In this chapter, we have learned why it is important to monitor models for degraded performance. To illustrate this idea, we used a synthetic dataset that captures ad-click behavior for mobile app downloads. First, we explored the data to understand the relationship between app downloads and ad-clicks. Then, we created features by aggregating existing click attributes in multiple dimensions. Next, we created three different datasets on which to run three experiments to explain the idea of model performance deterioration as new data becomes available. Next, we fitted the XGBoost model for each of the experiments. Finally, we evaluated performance across all the experiments to conclude that the model with the best performance is the one that took into account the latest and greatest click behavior.
Consequently, implementing a feedback loop in a machine learning life cycle...
