Data mesh governance requirements

One of the primary ideas behind a data mesh architecture is the concept of data products. Teams working on these data products need to be able to find good quality data, curate it, process it, and make the output available to anybody who can benefit from it and is authorized to use it. Similar to a software product, the team is also responsible for providing a service-level agreement (SLA) on their output so that the consumers of the data product know how reliable the output is.

For the various teams working on a data mesh to be able to work efficiently, we need good, streamlined governance of data and the underlying infrastructure.

Let’s consider a sample workflow in a data mesh environment to understand the importance of governance.

A team of data scientists want to build a sales-forecast data product that applies a machine learning algorithm to enterprise sales data to provide a more accurate forecast. They are looking for good-quality...