Cross-validation
Cross-validation is a resampling method as well, similar to the jackknife. However, the aim is now not to make inference statistics but to estimate prediction errors.
Cross-validation is mainly used for the comparison of methods or to find the optimal values of parameters in an estimation model.
In the following section, we will explain cross-validation based on regression analysis. For readers who have never heard of regression analysis, we recommend to read a basic textbook about regression analysis. We only point out some very basic issues.
The classical linear regression model
The classical linear regression model in its simplest case with one response and one predictor is given by 
with 
. In matrix notation, this is
, with the response y a vector of values, design matrix X with observations and p + 1 variables (including a vector of ones in the first column for the intercept term), 
a vector of size p + 1 and error term 
of length n. To keep it simple, we consider only...
