Detecting drift programmatically
With a comprehensive understanding of drift types and their effects, we will explore techniques for detecting drift programmatically, diving into the realms of concept drift and data drift. Armed with these methods, you’ll be well equipped to implement high-risk drift detection components. Let’s start with concept drift.
Detecting concept drift programmatically
Concept drift involves both the input data and the target data. This means that we can effectively detect concept drift for a deployed model only when we can get access to the real target labels in production. When you do have access to them, you can adopt the following techniques to detect concept drift:
- Check the similarity of production data to the reference training data: This should include both input and output data.
- Use model evaluation metrics as a proxy: Evaluation metrics can signal concept drift or data drift.
- Use multivariate-based data drift detection...
