Chapter 7. Understanding and Optimizing Data Flows

In this chapter, we will cover the following topics:

• Splitting a stream into two or more streams based on a condition

• Merging rows of two streams with the same or different structure

• Adding checksums to verify datasets

• Comparing two streams and generating differences

• Generating all possible pairs formed from two datasets

• Joining two or more streams based on given conditions

• Interspersing new rows between existent rows

• Executing steps even when your stream is empty

• Processing rows differently based on the row number

• Processing data into shared transformations via filter criteria and subtransformations

• Altering a data stream with Select values

• Processing multiple jobs or transformations in parallel