Summary

We began this chapter by briefly thinking about why boosting works. There are three perspectives that possibly explain the success of boosting, and these were covered before we looked deeper into this topic. The gbm package is very powerful, and it offers different options for tuning the gradient boosting algorithm, which deals with numerous data structures. We illustrated its capabilities with the shrinkage option and applied it to the count and survival data structures. The xgboost package is an even more efficient implementation of the gradient boosting method. It is faster and offers other flexibilities, too. We illustrated using the xgboost function with cross-validation, early stopping, and continuing further iterations as required. The h2o package/platform helps to implement the ensemble machine learning techniques on a bigger scale.

In the next chapter, we will look into the details of why ensembling works. In particular, we will see why putting multiple models together...