In this chapter we learned how Elixir elegantly tackles the problem of processing events in stages by analyzing the GenStage and Flow abstractions:
- We implemented our media upload and download pipelines with GenStage, putting to good use its elegant back-pressure mechanism. To achieve this, we had to implement the needed callbacks on the different modules behind the pipeline stages and start each pipeline supervisor on the application initialization.
- We then examined Flow, an abstraction based on GenStage that lets you process collections in a lazy and concurrent fashion, while keeping much of the semantics one associates with the Enum and Stream modules.
The advantage of using GenStage is that no step in the media pipeline will ever be overwhelmed with requests coming from upstream. If you want to process large datasets...
