The motivation behind model monitoring
According to an article in Forbes magazine by Enrique Dans, July 21, 2019, 87% of data science projects never make it to production (https://www.forbes.com/sites/enriquedans/2019/07/21/stop-experimenting-with-machine-learning-and-start-actually-usingit/?sh=1004ff0c3365).
There are a lot of reasons why ML models fail; however, if we look purely at the reason for ML project failure in a production environment, it comes down to a lack of re-training and testing the deployed models for performance consistency over time.
The performance of the model keeps degrading over time. Many data scientists neglect the maintenance aspect of the models post-production. The following visualizations offer a comparative analysis between two distinct approaches to model management—one where the model is trained once and then deployed for an extended period and another where the model undergoes regular retraining with fresh data while being monitored for...
