Monitoring and Data Observability
Monitoring and data observability are two sides of the same coin and are both critical for a well-managed data mesh.
Monitoring is all about proactively monitoring the health of the landing zones. As the data mesh grows and more data products start leveraging the mesh, it becomes critical to ensure that all the data products and their landing zones are up and running. This requires the proactive and reactive monitoring of the entire data mesh from a central location.
Data observability has more to do with a data mesh inherently being a collaborative framework. Data will be moving between landing zones and it’s very important to observe this movement of data and any changes made to the data.
In this chapter, we will see how to think about, design, and build a monitoring system from the ground up that will make the management of the data mesh easier.
In this chapter, we will cover the following:
- The importance of data mesh...
