Implementing a Streaming Use Case with Azure
Stream processing systems, or real-time processing systems, are systems that perform data processing in near real time. Think of stock market updates, real-time traffic updates, and real-time credit card fraud detection. Incoming data is processed as and when it arrives with very minimal latency, usually in the range of milliseconds to seconds. This section primarily focuses on the Create a stream processing solution by using Stream Analytics and Azure Event Hubs concept of the DP-203: Data Engineering on Microsoft Azure exam.
The streaming pipeline, as depicted by the gray boxes in Figure 6.1, involves several key stages:
- Real-time ingestion: This is where data is captured and imported into the system.
- Stream processing: This involves the real-time analysis and manipulation of the ingested data.
- Analytical data store: This is where processed data is stored for further analysis.
- Analytics and reporting: This is the...
