Introduction

Streaming has been picking up traction lately. It is one way of processing your data. In fact, there are two modes in which your data processing application can operate: batching and streaming. In batching, you work on batches of datasets at frequent intervals. However, in streaming, you process data as it gets streamed. This mode has always been a challenge. Achieving this type of streaming behavior will open up a lot of different opportunities.

You could make things happen quickly. For example, a banking application can send you relevant coupons based on your spending activity. Or a shopping application can recommend products based on your viewing activity. And all this will happen right away instead of waiting until the middle of the night for a batch job to run. Streaming has become a critical part of many businesses these days. The past couple of years have seen a tremendous improvement in streaming technologies. Frameworks such as Apex, Storm, Spark, Flink, Kafka Streams...