The bias-variance trade-off
Figure 8.9: The two extremes of the bias-variance tradeoff:. (left) a (complicated) model with essentially zero bias (on training data) but enormous variance, (right) a simple model with high bias but virtually no variance
In statistical learning, the bias of a model refers to the error of the model introduced by attempting to model a complicated real-life relationship with an approximation. A model with no bias will never make any errors in prediction (like the cookie-area prediction problem). A model with high bias will fail to accurately predict its dependent variable.
The variance of a model refers to how sensitive a model is to changes in the data that built the model. A model with low variance would change very little when built with new data. A linear model with high variance is very sensitive to changes to the data that it was built with, and the estimated coefficients will be unstable.
The term bias-variance tradeoff illustrates that it is easy to decrease...
