Stack ensembling

An introductory and motivational example of the stacked regression was provided in Chapter 1, Introduction to Ensemble Techniques. Here, we will continue the discussion of stacked ensembles for a regression problem which has not been previously developed.

With stacked ensembling, the outputs of several weak models are given as an input variable, along with the covariates used to build the earlier models, to build a stack model. The form of the stack model might be one of these, or it can be a different model. Here, we will simply use the eight regression models (used in previous sections) as weak models. The stacking regression model is selected as the gradient boosting model, and it will be given the original input variables and predictions of the new models, as follows:

> SP_lm_train <- predict(SP_lm,newdata=ht_imp)
Warning message:
In predict.lm(SP_lm, newdata = ht_imp) :
  prediction from a rank-deficient fit may be misleading
> SP_rpart2_train <- predict...