Building an End-to-End Analytics System – Lakehouse
Traditionally, for their analytics needs, companies have struggled to manage two different analytics systems: a relational data warehouse to manage and process primarily structured data and a data lake for big data processing (primarily unstructured data). This has not only created data silos and redundancy across multiple systems but has also increased the overall effort to develop and manage the increased total cost of ownership. Microsoft Fabric bridges this gap by unifying different data stores (data warehouses and data lakes) by standardizing data storage using the Delta Lake format in OneLake for lakehouses.
In this chapter, we are going to take an example of a retail organization and build its end-to-end analytics system based on a lakehouse from start to finish—all the way from data ingestion and transformation to reporting and visualization. The key stages are as follows:
- Creating a lakehouse using...
