Summary

In this chapter, you learned about the importance of ensemble methods. In particular, you learned about bagging, the combination of bootstrapping, sampling with replacement, and aggregation, combining many models into one. You built random forest classifiers and regressors. You adjusted n_estimators with the warm_start hyperparameter and used oob_score_ to find errors. Then you modified random forest hyperparameters to fine-tune models. Finally, you examined a case study where shuffling the data gave excellent results but adding more trees to the random forest did not result in any gains with the unshuffled data, as contrasted with XGBoost.

In the next chapter, you will learn the fundamentals of boosting, an ensemble method that learns from its mistakes to improve upon accuracy as more trees are added. You will implement gradient boosting to make predictions, thereby setting the stage for Extreme gradient boosting, better known as XGBoost.