Summary

We started this chapter by demonstrating, with several real-world examples, that regularization is the key to success in ML in a production environment. Along with several other methods and best practices, a robustly regularized model is necessary for production. In production, unseen data and edge cases will appear on a regular basis, thus any deployed model must have an acceptable response to such cases.

We then walked through some key concepts of regularization. Overfitting and underfitting are two common problems in ML and relate somehow to bias and variance. Indeed, an overfitting model has high variance, while an underfitting model has high bias. Thus, to perform well, a model is required to have low bias and low variance. We explained how, no matter how good a model can get, unavoidable bias limits its performance. Those key concepts allowed us to propose a method to diagnose bias and variance using the performance of both the training and validation sets, as well...