Scheduled monitoring with SageMaker Model Monitor

If you have been working in the data science and ML industry for quite some time, you probably know that an ML model’s performance after deployment is not guaranteed. Deployed models in production must be monitored in real time (or near-real time) so that we can potentially replace the deployed model and fix any issues once any drift or deviation from the expected set of values is detected:

Figure 8.9 – Analyzing captured data and detecting violations using Model Monitor

In the preceding diagram, we can see that we can process and analyze the captured data through a monitoring (processing) job. This job is expected to generate an automated report that can be used to analyze the deployed model and the data. At the same time, any detected violations are flagged and reported as part of the report.

Note

Let’s say that we have trained an ML model that predicts a professional’s salary...