There are some briefly described pre-requirements and useful considerations for a ‘real-time enterprise’ in this recent William McKnight’s post. To achieve real-time data warehouses, William also suggests rescheduling ‘batch’ ETL (Extract, Transform and Load) to run more frequently.

I’d add that it’s not about rescheduling frequency only; it’s about rescheduling the very moments of ETL operations. Sometimes it is not enough to increase the frequency, it even may be in vain. First, there’s a challenge of increased queue contentions. Second, there’s no use updating data, if there were no changes in it.

Some EAI advisors recommend using alert notifications instead. It is about defining the moments of updating as well as transforming data on an event basis, rather than according to a fixed schedule. However, it is clear that the real-time event model is more complicated, while developing. But as we see, EAI/ETL/EII vendors are already being forced to aiming at it, anyway.   

 

Filed under: Business Intelligence, Data Integration, Data Quality | No Comments »

Filed under: Data Integration, Data Quality | No Comments »