Ensemble of ensembles - model stacking

Ensemble of ensembles or model stacking is a method to combine different classifiers into a meta-classifier that has a better generalization performance than each individual classifier in isolation. It is always advisable to take opinions from many people when you are in doubt, when dealing with problems in your personal life too! There are two ways to perform ensembles on models:

• Ensemble with different types of classifiers: In this methodology, different types of classifiers (for example, logistic regression, decision trees, random forest, and so on) are fitted on the same training data and results are combined based on either majority voting or average, based on if it is classification or regression problems.
• Ensemble with a single type of classifiers, but built separately on various bootstrap samples: In this methodology, bootstrap samples are drawn from training data and, each time, separate models will be fitted (individual models could be decision...