Working with tree-based ensemble classifiers

Supervised tree-based ensemble classification and regression techniques have proved very successful in many practical real-world applications in recent years. Hence, they are widely used today in various applications such as fraud detection, recommendation engines, tagging engines, and many more. Your favorite OS (mobile and desktop), office program, and audio or video streaming service will use them heavily every day.

Therefore, we will dive into the main reasons and drivers for their popularity and performance, both for training and scoring, in this section. If you are an expert on traditional ML algorithms and know the difference between boosting and bagging, you might as well jump right to the Training an ensemble classifier model using LightGBM section—otherwise, I encourage you to read this section carefully.

We will first look at decision trees, a very simple technique that is decades old. I encourage you to follow along...