Advanced Boosting Algorithms
In this chapter, we are going to discuss some important algorithms that exploit different estimators to improve the overall performance of an ensemble or committee. These techniques work either by introducing a medium level of randomness in every estimator belonging to a predefined set, or by creating a sequence of estimators where each new model is forced to improve the performance of the previous ones. These techniques allow us to reduce both the bias and the variance (thereby increasing validation accuracy) when employing models with a limited capacity or that are more prone to overfit the training set.
In particular, the topics covered in the chapter are as follows:
- Gradient boosting
- Ensembles of voting classifiers, stacking, and bucketing
We can now start the exploration of the main concepts related to gradient boosting, which is an extremely flexible model that exploits both the simplicity of simpler algorithms (like...
