Overfitting and Underfitting
Let's say we fit a supervised learning algorithm to our data and subsequently use the model to perform a prediction on a hold-out validation set. The performance of this model will be considered to be good based on how well it generalizes, that is, how well it makes predictions for data points in an independent validation dataset.
Sometimes, we find that the model is not able to make accurate predictions and gives poor performance on the validation data. This poor performance can be the result of a model that is too simple to model the data appropriately, or a model that is too complex to generalize to the validation dataset. In the former case, the model has a high bias and results in underfitting, while, in the latter case, the model has a high variance and results in overfitting.
Bias
The bias in the prediction of a machine learning model represents the difference between the predicted target value and the true target value of a data point...
