Summary
In this chapter, we have learned several strategies for monitoring the performance of predictive models following initial design and looked at a number of scenarios where the performance or components of the model change over time. As part of the process of refining models, we examined A/B testing strategies and illustrated how to perform basic random allocation and estimate the sample sizes needed to measure improvement. We also demonstrated how to leverage the infrastructure from our prediction service to create dashboard visualizations for monitoring, which can easily be extended for other use cases.
