Summary

In this chapter, we discussed two additional algorithms that may be used to solve tabular learning problems: XGBoost, another gradient-boosting framework, and TabTransformer, a deep learning approach.

We showed how to set up and train both XGBoost models and TabTransformer on two datasets. We also showed how to encode categorical features for tree-based and neural network models. Both datasets also had imbalanced classes, which we had to compensate for during training.

We found that LightGBM and XGBoost produced similarly accurate models but that LightGBM trained models much faster and more efficiently. We also saw the complexity of training DNNs and the lackluster performance on these problems. Deep learning is an extremely powerful technique, but tree-based approaches are often more applicable when working with tabular datasets.

In the next chapter, we focus on more effective parameter optimization with LightGBM using a framework called Optuna.