Ridge regression for linear models

In the figure Regression coefficients of polynomial regression models, we saw that the magnitude of the regression coefficients increase in a drastic manner as the polynomial degree increases. The right tweaking of the linear regression model, as seen in the previous section, gives us the right results.

However, the models considered in the previous section had just one covariate and the problem of identifying the knots in the multiple regression model becomes an overtly complex issue. That is, if we have a problem where there are large numbers of covariates, there may naturally be some dependency among them, which cannot be investigated for certain reasons.

In such problems, it may happen that certain covariates dominate other covariates in terms of the magnitude of their regression coefficients, and this may mar the overall usefulness of the model. Furthermore, even in the univariate case, we have the problem that the choice of the number of knots, their...