Overcoming traditional ETL systems architectural problems using a service-oriented approach

Abstract

Developing analytical systems imposes several challenges related not only to the amount and heterogeneity of the involved data but also to the constant need to readapt and evolve to overcome new business challenges. Data are a determinant factor in the success of analytical and decision-making applications, being its nature, availability, and quality, crucial aspects for planning and structuring populating analytical systems. Today's users are more demanding, requiring adaptable and flexible analytical applications, which impose serious challenges on extract-transform-load (ETL) systems design and development for ensuring flexible and robust data populating services, operating 24/7, and managing and processing large volumes of data. Thus, we should design and implement ETL processes using innovative and up-to-date approaches, having real application evidence. In this paper, we present a service-oriented implementation for ETL design and development. We mapped and implemented some of the most conventional ETL processes in a service-oriented architecture, to demonstrate the application and benefits that this kind of approach will provide to ETL systems project development.