The application is therefore logically structured in components,
and single components are then organized/implemented in processUnits.
This two-level modularization is also useful for operations and maintenance issues
as described later.
Extraction, Transformation, and Loading Processes 0
Copyright ?© 2007, Idea Group Inc. Copying or distributing in print or electronic forms without written permission
of Idea Group Inc. is prohibited.
Main-Memory.Support
In any ETL application, a main-memory support is essential because of lookup operations.
In a fact table, there are always some foreign keys, and these must be set
in the loading phase with the corresponding value in the dimensional table. This
operation is very simple and looks like ???select key from tab where value = ??¦??? but
must be repeated many, many times: 10 foreign key and 10 millions rows to load
implies 100 millions of ???select.??? It is clear that these operations cannot be performed
on DBMS. The solution is to bring the dimensional tables into memory, indexing
them (with simple hash structure or binary search) and so perform the lookup operation
without involving DBMS.
These main-memory functions are adequate for simple cases, but often a more sophisticated
support should be useful. Sometimes a dimensional table must be updated
in consequence of information contained in fact records; the same dimensional
tables are big and it would be useful to manage them in memory; the application
logic imposes some processing of temporary data in memory, and so on.
Pages:
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211