Chapter 14: The Data Lakehouse

Throughout this book, you have encountered two primary data analytics use cases: descriptive analytics, which includes BI and SQL analytics, and advanced analytics, which includes data science and machine learning. You learned how Apache Spark, as a unified data analytics platform, can cater to all these use cases. Apache Spark, being a computational platform, is data storage-agnostic and can work with any traditional storage mechanisms, such as databases and data warehouses, and modern distributed data storage systems, such as data lakes. However, traditional descriptive analytics tools, such as BI tools, are designed around data warehouses and expect data to be presented in a certain way. Modern advanced analytics and data science tools are geared toward working with large amounts of data that can easily be accessed on data lakes. It is also not practical or cost-effective to store redundant data in separate storage to be able to cater to these individual...