Chapter 7. Job Orchestration

In the previous chapters, we have looked at examples of integration jobs that had a single purpose and were composed of small numbers of components, maybe two or three together. The purpose of these examples was to explain how different components work and are configured; but, in real-life situations, our integration jobs are likely to be made up of 10, 20, or more components, as we use Talend integration jobs to control the full end-to-end process, from file management to data processing tasks; from file mapping to database operations.

In this chapter, we will learn about job orchestration and how we can use subjobs, iterate connections, and flow components to control the logical flow of our jobs.

We will learn the following topics in this chapter:

• Breaking the overall integration job into discrete tasks using subjobs

• Defining flow logic within our jobs so that certain tasks are invoked only if specific conditions apply

• Providing checkpoints where we can trap errors...