Exploring LightGBM hyperparameters
Light Gradient Boosting Machine (LightGBM) is also a boosting algorithm built on top of a collection of decision trees, similar to XGBoost. It can also be utilized both for classification and regression tasks. However, it differs from XGBoost in the way the trees are grown. In LightGBM, trees are grown in a leaf-wise manner, while XGBoost grows trees in a level-wise manner (see Figure 11.2). By leaf-wise, we mean that LightGBM grows trees by prioritizing nodes whose split leads to the highest increase of homogeneity:
Figure 11.2 – Level-wise versus leaf-wise tree growth
Besides the difference in how XGBoost and LightGBM grow the trees, they also have different ways of handling categorical features. In XGBoost, we need to encode the categorical features before passing them to the model. This is usually done using the one-hot encoding or integer encoding methods. In LightGBM, we can just tell which features are categorical...