A common way to solve this problem is to use an
OCI wrapper, in other words, to build a set of functions suited for its own purposes
over the OCI layer (some wrappers are also available via the Internet).
Massive loading into a database may impose heavy constraints; indexes must be
inactive, referential constraints cannot be used, particular criterion in extents allocation
occurs, and so forth. Not all these limits can be escaped because the use of
traditional SQL insert is not suited for massive loading (millions of rows and more).
In ETL, one can use SQL only for managing dimension or loading fact in NRT
with small volumes. Massive loading does not generally support commit/rollback
statements, so one needs to build one??™s own error checks and perform rollback by
hand. How to manage and recover errors, in loading, but even in other cases, will
be discussed in the ???Operation and Maintenance Issue??? section.
Transform and Load Hub
Acquisition, especially transform and load, are operations that involve I/O and CPUbound
operations tightly correlated with each other and so they are well suited to be
parallelized for performance. Only in very special cases, the data must be processed
in sequential order; normally, records need to be transformed and loaded in relational
tables in an independent way. This assumption makes a single row parallelization
possible; each row can be acquired, transformed, and loaded independently, having
many pipelines in parallel.
Pages:
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218