Consequently, a major problem arises with the scheduling of the overall process.
The administrator has to find the right execution order for dependent jobs
and job sets on the existing hardware for the permitted time schedule. On the
other hand, if the OLTP applications cannot produce the necessary source data
in time for processing before the data warehouse comes online, the information
in the data warehouse will be out of date. Still, since data warehouses are
used for strategic purposes, this problem can sometimes be afforded, due to
the fact that long-term reporting/planning is not severely affected by this type
of failures.
??? Extraction.and.transportation:.During the ETL process, one of the very first
tasks that must be performed is the extraction of the relevant information that
has to be further propagated to the warehouse (Theodoratos, Ligoudistianos,
& Sellis, 2001). In order to minimize the overall processing time, this involves
only a fraction of the source data that has changed since the previous execution
of the ETL process, mainly concerning the newly inserted and possibly updated
records. Usually, change detection is physically performed by the comparison
of two snapshots (one corresponding to the previous extraction and the other
to the current one). Efficient algorithms exist for this task, like the snapshot
differential algorithms presented by Labio and Garcia-Molina (1996).
Pages:
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246