Tuning the model

During the model validation process, we evaluate the model performances, and there are situations where the model does not fit the validation dataset. Let’s examine the different cases.

Overfitting and underfitting

While underfitting describes the situation where prediction error is not minimized, overfitting is the case where the model fits the training dataset very well but does not fit the validation dataset. An overfitting model gets a very low cost function value during training but poorly predicts on new data. Figure 4.16 depicts the situations for underfitting, robust, and overfitting.

Figure 4.16 – Model fittings

When we try to minimize the cost function and avoid underfitting, we need to make sure our model is generalized and not prone to overfitting. From our ML practice, we know that overfitting is often caused by making a model more complex than necessary. As you can see in Figure 4.16, overfitting makes a training...