Underfitting and Overfitting

In the previous scenario, where we have a high bias, we denote a phenomenon called underfitting in machine learning models. Similarly, when we have high variance, we denote a phenomenon called overfitting in machine learning models.

The following visual demonstrates the idea of overfitting, underfitting, and ideal balance for a regression model. We can see high bias resulting in an oversimplified model (that is, underfitting); high variance resulting in overcomplicated models (that is, overfitting); and lastly, striking the right balance between bias and variance:

Figure 7.2: Visual demonstration of overfitting, underfitting, and ideal balance

To study bias and variance in machine learning models more effectively, we have cross-validation techniques. These techniques help us understand the model performance more intuitively.