Summary

In this chapter, you have learned about big data architectural patterns such as Lambda and Kappa for historical and interactive complex stream processing along with in-depth analysis of batch processing and the speed and serving layer for ad hoc querying. In the real world, the big and fast data processing pipeline follows mostly Lambda or Kappa design patterns from events ingestion to processing and finally implementing near real-time intelligent visual dashboards. We have provided step-by-step guidance of developing a real-time visual dashboard using Microsoft Power BI with processed data from Azure Stream Analytics as the output data connector.

In the next chapter, we will be concentrating on designing and managing Stream Analytics jobs using reference data and utilizing petabyte-scale enterprise data store with Azure Data Lake Store and a globally distributed NoSQL database from Microsoft Azure Cosmos DB—and the next generation server-less cloud architectures with Azure Functions...