Monitoring and managing data quality

The data mesh is all about converting analytics in data products, bringing the application development paradigm to data, and operationalizing data and analytics just like DevOps in the application world. One of the key aspects of application development is quality assurance (QA). The same principles will also apply to data products. Data products are designed to be consumed by other data products or end users. These data products and end users will rely on the quality and availability of this data. Bad data can have a domino effect in a data mesh, with multiple products failing to deliver because of the unacceptable quality of one dataset.

Data quality management is an important aspect of data governance. It is an involved process and begins with defining the data quality standards. To define data quality standards, a company can form its own standards through a cross-organization standards committee or borrow a framework from standard organizations...