Exploring CatBoost hyperparameters

Categorical Boosting (CatBoost) is another boosting algorithm built on top of a collection of decision trees, similar to XGBoost and LightGBM. It can also be utilized both for classification and regression tasks. The main difference between CatBoost and XGBoost or LightGBM is how it grows the trees. In XGBoost and LightGBM, trees are grown asymmetrically, while in CatBoost, trees are grown symmetrically so that all of the trees are balanced. This balanced tree characteristic provides several benefits, including the ability to control overfitting problems, lower inference time, and efficient implementation in CPUs. CatBoost does this by using the same condition in every split in the nodes, as shown in the following diagram:

Figure 11.3 – Asymmetric versus symmetric tree

The main selling point of CatBoost is its ability to handle numerous types of features automatically, including numerical, categorical, and text, especially...