Ensemble diversity

In an ensemble, we have many base models—say L number of them. For the classification problem, we have base models as classifiers. If we have a regression problem, we have the base models as learners. Since the diagnostics are performed on the training dataset only, we will drop the convention of train and valid partitions. For simplicity, during the rest of the discussion, we will assume that we have N observations. The L number of models implies that we have L predictions for each of the N observations, and thus the number of predictions is . It is in these predictions that we try to find the diversity of the ensemble. The diversity of the ensemble is identified depending on the type of problem we are dealing with. First, we will take the regression problem.

Numeric prediction

In the case of regression problems, the predicted values of the observations can be compared directly with their actual values. We can easily see which base models' predictions are closer to the...