Continuous Model Evaluation
In this chapter, we will discuss continuous model evaluation, our second model serving pattern based on the model serving principle. Whenever we serve a model, we need to keep in mind that this model will underperform after a certain period. As data is growing continuously and new data patterns appear regularly, a model that has been trained on old data will very soon become stale. In this chapter, we will introduce you to continuous model evaluation, why it is needed, the techniques of continuous model evaluation, and example scenarios. At a high level, we will discuss the following topics:
- Introducing continuous model evaluation
- The necessity of continuous evaluation
- Continuous model evaluation use cases
- Evaluating a model continuously
- Monitoring model performance when predicting rare classes
