Advanced Techniques for Ensemble Learning

Having learned simple techniques for ensemble learning, let's now explore some advanced techniques. Among the advanced techniques, we will be dealing with three different kinds of ensemble learning:

• Bagging
• Boosting
• Stacking/blending

Before we deal with each of them, there are some basic dynamics of these advanced ensemble learning techniques that need to be deciphered. As described at the beginning of the chapter, the essence of ensemble learning is in combining individual models to form a superior model. There are some subtle nuances in the way the superior model is generated in the advanced techniques. In these techniques, the individual models or learners generate predictions and those predictions are used to form the final predictions. The individual models or learners, which generate the first set of predictions, are called base learners or base estimators and the model, which is a combination of the predictions...