Pipeline

We have already learned a lot about pipelines in the previous chapter. To recap, pipelines are useful when we can split data processing into multiple independent stages and run these stages in parallel, on multiple background threads. We also looked into implementing a pipeline with standard PPL tools (and some additional code), and while the solution is useful, it feels quite clumsy (which is partially my fault, as I tried to keep the PPL pipeline implementation as lightweight as possible).

In this section, we will look into the OTL implementation of pipelines and explore it through the ParallelPipeline demo. Although this demo has the same name as the corresponding demo in the previous chapter, it solves a different problem. In the previous chapter, we wrote a web-scanning spider. This time, the demo program implements a very simple file analyzer.

The ParallelPipeline demo scans all files with the .pas extension in a folder (including subfolders), counts the number...