Scaling Resources
Recall the example of credit card processing described at the beginning of the chapter involving the processing of credit card transactions in real time. The buyer and the seller will need to know as quickly as possible whether the transaction was approved. This was an example of stream processing.
Note
This section primarily focuses on the Scale resources concept of the DP-203: Data Engineering on Microsoft Azure exam.
In contrast to stream processing, batch processing involves processing a great quantity of data and executing at longer intervals of time, so it involves a steep spike in the resources required for its execution. Therefore, the need for scaling arises to efficiently manage and adapt the allocation of resources to accommodate these fluctuations in demand. Fortunately, cloud technology makes scaling much easier, and that applies to all the services covered in this chapter.
ADF and Synapse Pipelines
As a pipeline is just a sequence of activities...
