Overfitting and Underfitting
Let's say we fit a supervised learning algorithm to our data and subsequently use the model to perform a prediction on a hold-out validation set. The performance of this model will be considered to be good based on how well it generalizes, that is, the predictions it makes for data points in an independent validation dataset.
Sometimes we find that the model is not able to make accurate predictions and gives poor performance on the validation data. This poor performance can be the result of a model that is too simple to model the data appropriately, or a model that is too complex to generalize to the validation dataset. In the former case, the model has a high bias and results in underfitting, while in the latter case, the model has a high variance and results in overfitting.
Bias
The bias in the prediction of a machine learning model represents the difference between the predicted values and the true values. A model is said to have a high bias if the average predicted...
