Chapter 9: Scaling the Data Lakehouse Architecture

In the journey so far, we have covered all the seven layers of the data lakehouse architecture. However, for large organizations that are complex and spread globally, a single data lakehouse won't suffice. They will need multiple platforms to fulfill their analytical requirements. In addition, they will need a structured process to share data elements between them. Therefore, the need arises to develop a macro-architecture pattern that ensures that the organization's overall analytical requirements are met without compromising the architectural debts.

This chapter will discuss concepts that you can use to develop those macro-architecture patterns. First, we will define the need for a macro-architecture pattern for a data lakehouse and the organizational drivers that dictate the requirements for such a pattern. Next, we will cover two types of macro-architecture patterns, namely hub-spoke and data mesh. This section will...