Defining packages and loops for near real-time execution using a hybrid loop

There are many use cases where a specific job may require execution on a continual basis, perhaps the most common of which is when using a changed-data-capture process. A commonly occurring situation in Changed Data Capture (CDC) environments is the requirement that the ELT processes continuously monitor for data changes on the source system, rather than simply executing on a fixed schedule.

Unfortunately, a perpetual ELT process presents some logistical problems such as how to efficiently manage the job logs in the Operator navigator and how to afford an administrator the opportunity to terminate and restart the job. For example, if an ODI scenario is designed to recursively repeat an internal loop, all process activities will be collected into a single, enormous job log. On the other hand, if an ODI scenario is designed to reinvoke itself upon completion of its assigned tasks, the proliferation of individual job...