Summary
In this chapter, you have seen how catastrophic forgetting can cause bad performance in your model, especially when data arrives in a sequential manner. Especially when one trend is learned first and a second trend follows, the risk of forgetting the first trend is real and needs to be controlled.
Although there is no one-stop solution for these issues, there are many things that can be done to avoid bad models from going into production systems. You have seen how to implement continuous evaluation metrics and you have seen how you would be able to detect that some trends have been forgotten.
Performance-based metrics are great for detecting problems but are not able to tell you what exactly has gone wrong inside the model. You have seen three methods of model explanation that can help you deep-dive further into most models. By extracting from the model which trends or relationships the model has learned, you can identify whether this corresponds to an already known business...
