Chapter 11: Statistics for Ensemble Methods

In this chapter, we are going to investigate the ensemble method in terms of statistics and machine learning. The English word ensemble means a group of actors or musicians that work together as a whole. The ensemble method, or ensemble learning in machine learning, is not a specific machine learning algorithm, but a meta learning algorithm that builds on top of concrete machine learning algorithms to bundle them together to achieve better performance.

The ensemble method is not a single method, but a collection of many. In this chapter, we will cover the most important and representative ones.

We are going to cover the following in this chapter:

• Revisiting bias, variance, and memorization
• Understanding the bootstrapping and bagging techniques
• Understanding and using the boosting module
• Exploring random forests with scikit-learn

Let's get started!