The thin table is massively
loaded with the mentioned method of the temporary table exchange and,
periodically, the thin partitions will be compacted together in a new partition of the
fat table. With this method, complex only in appearance, we can contain the number
of partitions to a reasonable value, loading data with the most efficient techniques,
and defer the time-consuming compact operations to the optimal time window. All
these operations can be done while querying the data; the view ensures the transactional
consistency.
Pipelining.and. Parallelism.................
for.Performance.
Performance is one of the most important issues in ETL processing. Due to a high
volume of data, NRT time constraints, or hardware limits (not even the budget for
Figure 3. Thin-fat partitioning schema
FAT
THIN
p < 0 apr
p2 < may
p2 < jun
p < jun
TABLE (v ew)
p < 0 jun
p < jul
TEMP
Extraction, Transformation, and Loading Processes
Copyright ?© 2007, Idea Group Inc. Copying or distributing in print or electronic forms without written permission
of Idea Group Inc. is prohibited.
a DW project is large enough), it is always necessary to pay attention to the effi-
ciency of code.
Loading records into a DBMS with some transformations is not a complex job;
complexity results from a very great number of times that these simple operations
must be repeated.
Pages:
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203