An example of pseudocode for the transformation hub
Extraction, Transformation, and Loading Processes 0
Copyright ?© 2007, Idea Group Inc. Copying or distributing in print or electronic forms without written permission
of Idea Group Inc. is prohibited.
to a DBMS problem, an always possible application bug, or a hardware failure; but
above all, a loading process can fail due to an external problem: absence or corruption
of data, unavailability of some source systems, and so forth. In any case,
a failure anywhere in the process causes a corruption, data inconsistency, or data
loss. But we have to be straight; sometimes a loss of data is allowed, sometimes
not. Sometimes one can skip a loading phase and retry it later, and sometimes data
must be loaded without ???holes.??? DW requirements, even in this perspective, are
wide-ranging, so one system can meet its requirements almost ignoring the O&M
issues, and for another stability and ???fault-tolerance??? are mandatory.
Even in a quiet scenario like a population in nightly window, when in the morning
one sees that a failure has occurred, it is really hard to detect the bug, localize
the corrupted or incomplete data, clean them, and restart the application at the old
point-in-time. It is hard because one is under stress, the time in hand is constrained,
and the users claim to access the DW.
Pages:
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222