Ensemble Learning

Ensemble learning, as the name denotes, is a method that combines several machine learning models to generate a superior model, thereby decreasing variability/variance and bias, and boosting performance.

Before we explore what ensemble learning is, let's look at the concepts of bias and variance with the help of the classical bias-variance quadrant, as shown here:

Figure 15.1: Bias-variance quadrant

Variance

Variance is the measure of how spread out data is. In the context of machine learning, models with high variance imply that the predictions generated on the same test set will differ considerably when different training sets are used to fit the model. The underlying reason for high variability could be attributed to the model being attuned to specific nuances of training data rather than generalizing the relationship between input and output. Ideally, we want every machine learning model to have low variance.

Bias

Bias...