The main focus of our exposition here is the practical
application of the ETL process in real world cases with extra problems and strong
requirements, particularly performance issues related to population of large data
warehouses (one case study is described in Adzic & Fiore, 2003).
In this chapter, we will first discuss the ETL scenario, requirements, criticalities,
and so forth that constitute the general framework for ETL processes. Then we will
describe some techniques related to the physical database design, pipelining, and
parallelism which are relevant for performance issues. Finally, we will describe our
practical approach, ???infrastructure based ETL???; it is not a tool but a set of functionalities
or services that experience has proved to be useful and widespread enough
in the ETL scenario, and one can build the application on top of it.
ETL. Scenario
The primary scenario in which ETL takes place is a wide area between the sources
of data and a target database management system (DBMS); in the middle, there are
all the required functionalities to bring and maintain historical data in a form well
suited for analysis (Figure 1).
All the work to collect, transform, and load data from different and multiple sources
to a target DBMS structured for analysis is what we call ETL.
A DW project consists of three main technical tasks: ETL, database design, and
analysis techniques and tools; each of them has particular issues and requirements.
Pages:
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189