Chapter 6. Avoiding Overfitting Problems - Achieving Generalization

In the previous chapters, we have emphasized the importance of the training phase for successful modeling. In the training phase, the model is developed by accurately specifying the level of detail that the system will be able to predict. The higher the degree of detail required, the greater the ability to predict from the model. So far, nothing strange has been found. Problems arise when we use that model to make new predictions based on data that the model does not know. The risk we run is that we push the precision in the details so much that we lose the ability to generalize.

Let's consider a practical example: suppose we build a face recognition model. Since each pixel can be compared between one image and the other, it may happen that minor details become overwhelming: hair, background, shirt color, and so on. The number of details on which the model can play is so wide that it is able to identify individual images...