Tree-Based Machine Learning Models

We've seen a few of the simpler machine learning models, and now it's time to examine some more advanced models. In this chapter, we will look at the family of machine learning models that is based on decision trees. These models, especially the boosted models, have won machine learning contests and are used in industry for state-of-the-art ML performance. Here, we'll cover:

• How decision trees work in machine learning
• Random forests in sklearn and H2O, which are collections of decision trees
• Feature importances from tree-based methods
• Boosted algorithms, including AdaBoost, XGBoost, LightGBM, and CatBoost

Let's start with the basic decision tree and how it works.