Preface

In 2019, Zhamak Dehghani published her whitepaper on data mesh during her time at Thoughtworks. While it caught the attention of many large corporations, adopting data mesh was not easy. Most large companies have a strong legacy of analytical systems, and migrating them to a mesh architecture can be a daunting task. At the same time, the theoretical concepts of data mesh can be confusing when you map them to an actual analytical system.

In 2021, I started working with a large Microsoft customer that was struggling with their centralized data analytics platform. The platform was based on a central data lake and a single technology stack. It was rigid and was hard for all the stakeholders to adopt. As a result, many projects were creating their own siloed infrastructure, producing islands of data, technology, and expertise. We observed the dilemma the central analytics team was facing and proposed the data mesh architecture. It seemed that data mesh would solve most of their challenges around agility and adoption, as well as opening the doors to some other challenges, such as federated governance.

In the next year, we helped onboard this customer to data mesh. It was a long journey of multiple workshops followed by a consulting engagement where we built data mesh artifacts for them. Since then, I have been engaged with multiple customers on data mesh projects. As a member of a team of subject-matter experts on data mesh at Microsoft Europe, I have also guided other Microsoft team members on how to engage, design, and manage a data mesh project.

Along the way, I have realized that translating the theory of data mesh into a practical, production-ready system can be a challenge. A lot of terms get thrown around that actually can represent large projects in themselves.

This book consolidates information on all the challenges (and their solutions) involved in implementing data mesh on Microsoft Azure, going from understanding data mesh terminology and mapping it to Microsoft Azure artifacts to all those unknown things that only get mentioned as topics for you to look up for yourself in other data mesh resources. Some of these topics, such as master data management, data quality, and monitoring, can be large, complex systems in themselves.

The driving motivation behind writing this book is to help you understand the concepts of data mesh and to dive into their practical implementation. With this book, you will focus more on the benefits of a decentralized architecture and apply them to your own analytical landscape, rather than getting caught up in all the data mesh terminology.