How do we link data contracts to data consumption or pipelines?
Hosting contracts in a central repository that data owners and data consumers can access and maintain is just one-half of the solution. Sticking to these contracts and ensuring that data pipelines are not failing because they are using the wrong version of the data completes the end-to-end data contract implementation. We also need to ensure that this consistency check is done in an automated fashion so that the pipelines or programs are aware of the consistency and take the necessary actions if they observe a mismatch.
The first step in this process is to ensure that you have programmatic access to the data contracts. Other than providing read-and-write access to the data contract, you also have to allow users to browse and search the contract with keywords.
As mentioned in the What are the contents of a data contract? section, certain attributes in the data catalog might overlap with the attributes that we are...
