6. Gradient Boosting, XGBoost, and SHAP Values

Overview

After reading this chapter, you will be able to describe the concept of gradient boosting, the fundamental idea underlying the XGBoost package. You will then train XGBoost models on synthetic data, while learning about early stopping as well as several XGBoost hyperparameters along the way. In addition to using a similar method to grow trees as we have previously (by setting max_depth), you'll also discover a new way of growing trees that is offered by XGBoost: loss-guided tree growing. After learning about XGBoost, you'll then be introduced to a new and powerful way of explaining model predictions, called SHAP (SHapley Additive exPlanations). You will see how SHAP values can be used to provide individualized explanations for model predictions from any dataset, not just the training data, and also understand the additive property of SHAP values.