You're reading from Practical Data Science with Python Learn tools and techniques from hands-on examples to extract insights from data

Product type Paperback

Published in Sep 2021

Publisher Packt

ISBN-13 9781801071970

Length 620 pages

Edition 1st Edition

Languages

Python

Tools

Excel

Concepts

Data Science

Author (1):

Nathan George

View More author details

Table of Contents (30) Chapters

Preface

1. Part I - An Introduction and the Basics

2. Introduction to Data Science FREE CHAPTER

3. Getting Started with Python

4. Part II - Dealing with Data

5. SQL and Built-in File Handling Modules in Python

6. Loading and Wrangling Data with Pandas and NumPy

7. Exploratory Data Analysis and Visualization

8. Data Wrangling Documents and Spreadsheets

9. Web Scraping

10. Part III - Statistics for Data Science

11. Probability, Distributions, and Sampling

12. Statistical Testing for Data Science

13. Part IV - Machine Learning

14. Preparing Data for Machine Learning: Feature Selection, Feature Engineering, and Dimensionality Reduction

15. Machine Learning for Classification

16. Evaluating Machine Learning Classification Models and Sampling for Classification

17. Machine Learning with Regression

18. Optimizing Models and Using AutoML

19. Tree-Based Machine Learning Models

20. Support Vector Machine (SVM) Machine Learning Models

21. Part V - Text Analysis and Reporting

22. Clustering with Machine Learning

23. Working with Text

24. Part VI - Wrapping Up

25. Data Storytelling and Automated Reporting/Dashboarding

26. Ethics and Privacy

27. Staying Up to Date and the Future of Data Science

28. Other Books You May Enjoy

29. Index

Boosted trees: AdaBoost, XGboost, LightGBM, and CatBoost

Boosted machine learning models were first introduced around 1989 and have been shown to perform well. Some of the more common boosting algorithms you will see are AdaBoost, gradient boosting, XGBoost, LightGBM, and CatBoost.

XGBoost has been used to win several machine learning competitions (for example, on Kaggle), and was initially released in 2014. LightGBM was developed shortly after by Microsoft and released in 2016, while CatBoost was released in 2017. (For a more detailed history of boosting, see this paper: https://cseweb.ucsd.edu/~yfreund/papers/IntroToBoosting.pdf.) These boosting algorithms have slightly different algorithms and implementations, and when trying models on a dataset, it doesn't hurt to try as many of them as you can. An easy way to do this is with the PyCaret package we covered in the previous chapter on model optimization and AutoML.

First, let's learn about how boosting works....