Retraining required
You wouldn’t expect that after finishing your education, you never read a paper or book or speak to anyone again, which would mean you wouldn’t be able to make informed decisions about what is happening in the world. So, you shouldn’t expect an ML model to be trained once and then be performant forever afterward.
This idea is intuitive, but it represents a formal problem for ML models known as drift. Drift is a term that covers a variety of reasons for your model’s performance dropping over time. It can be split into two main types:
- Concept drift: This happens when there is a change in the fundamental relationship between the features of your data and the outcome you are trying to predict. Sometimes, this is also known as covariate drift. An example could be that at the time of training, you only have a subsample of data that seems to show a linear relationship between the features and your outcome. If it turns out that...
