Regularization with Tree-Based Models
Tree-based models using ensemble learning such as Random Forest or Gradient Boosting are often seen as easy-to-use, state-of-the-art models for regular machine learning tasks.
Many Kaggle competitions have been won with such models, as they can be quite robust and efficient at finding complex patterns in data. Knowing how to regularize and fine-tune them is key to having the very best performance.
In this chapter, we’ll look at the following recipes:
- Building a classification tree
- Building regression trees
- Regularizing a decision tree
- Training a Random Forest algorithm
- Regularization of Random Forest
- Training a boosting model with XGBoost
- Regularization with XGBoost
