Colored Petri nets in the simulation of ETL standard tasks: the surrogate key pipelining case

Abstract

ETL (Extract-Transform-Load) systems are formed by processes responsible for the extraction of data from several sources, cleaning and transforming it in accordance with some prerequisites of a data warehouse, and finally loading it in its multidimensional structures. ETL processes are the most complex tasks involved within the development of a Data Warehousing System, being crucial to model them previously so that, during the implementation stage, the correct set of requirements is considered. Coloured Petri Nets are a graphical modelling language used in the design, specification, simulation and validation of large systems, characterized as being strongly concurrent. The objective of this manuscript is to discuss the application of Coloured Petri Nets to the specification and validation of ETL systems. To demonstrate their viability for such tasks we have selected one of the most relevant and used case in ETL systems implementation: a surrogate key pipelining.