Summary
Information is dynamic and constantly evolving, which is why success in business is based on how we make use of this continuously changing data.
The modern data platform is built on business-centric value chains rather than IT-centric coding pipelines. The focus is to provide insights faster by turning event streams into analytics-ready data. Stream processing naturally fits with time series data and supports the detection of patterns over time. For some scenarios, streaming is a must, for example – sensor data, advertisement data, server security logs, and clickstream data. In some others, it is not, but every batch job can be regarded as a streaming job with a longer trigger interval and because the dial is configurable, it can be tweaked to make it more real time in line with business demands without having to rewrite the pipeline.
In this chapter, we focused on stream processing for ingesting, processing, and storing data. In the next chapter, we will look...
