Doing linear regression with lasso

The lasso is a shrinkage and selection method for linear regression. It minimizes the usual sum of squared errors, with a bound on the sum of the absolute values of the coefficients. It is based on the original lasso paper found at http://statweb.stanford.edu/~tibs/lasso/lasso.pdf.

The least square method we used in the last recipe is also called ordinary least squares (OLS). OLS has two challenges:

• Prediction accuracy: Predictions made using OLS usually have low forecast bias and high variance. Prediction accuracy can be improved by shrinking some coefficients (or even making them zero). There will be some increase in bias, but overall prediction accuracy will improve.

• Interpretation: With a large number of predictors, it is desirable to find a subset of them that exhibits the strongest effect (correlation).

Note

Bias versus variance

There are two primary reasons behind prediction error: bias and variance. The best way to understand bias and variance is to look...