All Products

Best Sellers

New Releases

Books

Videos

Audiobooks

Free Learning

Hands-On Gradient Boosting with XGBoost and scikit-learn

You're reading from Hands-On Gradient Boosting with XGBoost and scikit-learn

Product type Book

Published in Oct 2020

Publisher Packt

ISBN-13 9781839218354

Pages 310 pages

Edition 1st Edition

Languages

Concepts

Machine Learning

Author (1):

Corey Wade

Table of Contents (15) Chapters

Preface

1. Section 1: Bagging and Boosting

2. Chapter 1: Machine Learning Landscape

3. Chapter 2: Decision Trees in Depth

4. Chapter 3: Bagging with Random Forests

5. Chapter 4: From Gradient Boosting to XGBoost

6. Section 2: XGBoost

7. Chapter 5: XGBoost Unveiled

8. Chapter 6: XGBoost Hyperparameters

9. Chapter 7: Discovering Exoplanets with XGBoost

10. Section 3: Advanced XGBoost

11. Chapter 8: XGBoost Alternative Base Learners

12. Chapter 9: XGBoost Kaggle Masters

13. Chapter 10: XGBoost Model Deployment

14. Other Books You May Enjoy

Leave a review - let other readers know what you think

Random forest hyperparameters

The range of random forest hyperparameters is large, unless one already has a working knowledge of decision tree hyperparameters, as covered in Chapter 2, Decision Trees in Depth.

In this section, we will go over additional random forest hyperparameters before grouping the hyperparameters that you have already seen. Many of these hyperparameters will be used by XGBoost.

oob_score

Our first hyperparameter, and perhaps the most intriguing, is oob_score.

Random forests select decision trees via bagging, meaning that samples are selected with replacement. After all of the samples have been chosen, some samples should remain that have not been chosen.

It's possible to hold back these samples as the test set. After the model is fit on one tree, the model can immediately be scored against this test set. When the hyperparameter is set to oob_score=True, this is exactly what happens.

In other words, oob_score provides a shortcut to get a...

The rest of the chapter is locked

Register for a free Packt account to unlock a world of extra content!

A free Packt account unlocks extra newsletters, articles, discounted offers, and much more. Start advancing your knowledge today.

Unlock this book and the full library FREE for 7 days

Get unlimited access to 7000+ expert-authored eBooks and videos courses covering every tech area you can think of

Start free trial

Renews at €14.99/month. Cancel anytime

Authors (1)

Corey Wade

Corey Wade, M.S. Mathematics, M.F.A. Writing & Consciousness, is the founder and director of Berkeley Coding Academy where he teaches Machine Learning and AI to teens from all over the world. Additionally, Corey chairs the Math Department at Berkeley Independent Study where he has received multiple grants to run after-school coding programs to help bridge the tech skills gap. Additional experiences include teaching Natural Language Processing with Hello World, developing Data Science curricula with Pathstream, and publishing statistics and machine learning models with Towards Data Science, Springboard, and Medium.

See other products by Corey Wade

Other recommended products

Related to this chapter

Machine Learning with scikit-learn Quick Start Guide

Machine Learning with scikit-learn Quick Start Guide

Scikit-learn is a robust machine learning library for the Python programming language. It provides a set of supervised and unsupervised learning algorithms. This book is the easiest way to learn how to deploy, optimize and evaluate all the important machine learning algorithms that scikit-learn provides.

Oct 2018 5 hours 44 minutes

The Data Analysis Workshop

The Data Analysis Workshop

The Data Analysis Workshop is a comprehensive guide that shows you how to analyze your data and gain insights into your business. Starting with the basics of data analysis, including data visualization and exploratory data analysis, this book takes you through the complete spectrum of techniques, such as time series analysis and categorical data analysis. It is the ideal companion on your journey to becoming an expert data analyst.

Jul 2020 20 hours 52 minutes

scikit-learn Cookbook

scikit-learn Cookbook

scikit-learn has evolved as a robust library for machine learning applications in python with support for a wide range of supervised and unsupervised learning algorithms. This edition brings to you the various enhancements to its model implementations, API and bug fixes in the latest major release of scikit-learn to support Python. This book covers easy to follow recipes right from mathematical operations to implementing various supervised, unsupervised and deep learning algorithms with scikit-learn. Get practical hands-on knowledge to implement various models and algorithms like Multi-Layer Perceptrons, time-series split, MAE criterion for regression, criteria for gradient boosting, Classifier, Regressor, and much more.

Nov 2017 12 hours 28 minutes

Master Data Science with Python

Master Data Science with Python

Data Science with Python will help you get comfortable with using the Python environment for data science. You will learn all the libraries that a data scientist uses on a daily basis. By the end of this course, you will be able to take a large raw dataset, clean it, manipulate it, and run machine learning algorithms to obtain results that influence business decisions.

Jul 2019 14 hours 12 minutes

Hands-On Machine Learning with scikit-learn and Scientific Python Toolkits

Hands-On Machine Learning with scikit-learn and Scientific Python Toolkits

This book covers the theory and practice of building data-driven solutions. Includes the end-to-end process, using supervised and unsupervised algorithms. With each algorithm, you will learn the data acquisition and data engineering methods, the apt metrics, and the available hyper-parameters. You will learn how to deploy the models in production.

Jul 2020 12 hours 48 minutes

Ensemble Machine Learning Cookbook

Ensemble Machine Learning Cookbook

This book uses a recipe-based approach to showcase the power of machine learning algorithms to build ensemble models using Python libraries. Through this book, you will be able to pick up the code, understand in depth how it works, execute and implement it efficiently. This will be a desk reference to implement a wide range of tasks and solve the common and uncommon problems in ensemble machine learning domain.

Jan 2019 11 hours 12 minutes

Data Science for Marketing Analytics

Data Science for Marketing Analytics

Data Science for Marketing Analytics opens doors to looking at data with a different approach and new tools. Drawing on machine learning and data science concepts, this book broadens the range of tools that you can use to transform the market analysis process.

Mar 2019 14 hours 0 minutes

Applied Deep Learning with Keras

Applied Deep Learning with Keras

Applied Deep Learning with Keras takes you from a basic knowledge of machine learning and Python to an expert understanding of applying Keras to develop efficient deep learning solutions. This book teaches you new techniques to handle neural networks, and in turn, broadens your options as a data scientist.

Apr 2019 13 hours 44 minutes

Data Science Projects with Python

Data Science Projects with Python

Ideal for anyone who is just getting started with machine learning, this hands-on data science book will give you experience building predictive models using industry-standard tools and techniques. It will help you develop the skills and understanding to generate valuable insights and make data-driven business decisions.

Jul 2021 14 hours 24 minutes

Data Science for Marketing Analytics

Data Science for Marketing Analytics

This book on marketing analytics with Python will quickly get you up and running using practical data science and machine learning to improve your approach to marketing. You'll learn how to analyze sales, understand customer data, predict outcomes, and present conclusions with clear visualizations.

Sep 2021 21 hours 12 minutes

Python Artificial Intelligence Projects for Beginners

Python Artificial Intelligence Projects for Beginners

This book demonstrates AI projects in Python covering modern techniques that make up the world of Artificial Intelligence. You will come across a variety of real-world projects on classifying data, text processing techniques, deep learning and neural networks

Jul 2018 5 hours 24 minutes

Hands-On Ensemble Learning with Python

Hands-On Ensemble Learning with Python

Ensemble learning can provide the necessary methods to improve the accuracy and performance of existing models. In this book, you'll understand how to combine different machine learning algorithms to produce more accurate results from your models.

Jul 2019 9 hours 56 minutes

Personalised recommendations for you

Based on your interests and search pattern

Ever wonder why speech recognition systems don't understand the Scottish accent, or what would happen if an astronaut only ate mac 'n' cheese, or other spurious reflections you'd have at a bar? We did, then collated those deliberations into absurd research articles with fake figures and methodologies inspired by even more fictionally absurd studies.

Aug 2023 7 hours 40 minutes

Generative AI with LangChain

Generative AI with LangChain

This book is a comprehensive introduction to LLMs and LangChain, demystifying the basic mechanics of LangChain, its functionalities, and the myriad of applications it can be integrated into.

Dec 2023 12 hours 0 minutes

Generative AI with LangChain

Generative AI with LangChain

This book is a comprehensive introduction to LLMs and LangChain, demystifying the basic mechanics of LangChain, its functionalities, and the myriad of applications it can be integrated into.

Dec 2023 12 hours 0 minutes

Generative AI with LangChain

Generative AI with LangChain

This book is a comprehensive introduction to LLMs and LangChain, demystifying the basic mechanics of LangChain, its functionalities, and the myriad of applications it can be integrated into.

Dec 2023 12 hours 0 minutes

Generative AI with LangChain

Generative AI with LangChain

This book is a comprehensive introduction to LLMs and LangChain, demystifying the basic mechanics of LangChain, its functionalities, and the myriad of applications it can be integrated into.

Dec 2023 12 hours 0 minutes

Mastering Tableau 2023

Mastering Tableau 2023

This book is a comprehensive resource to mastering your Tableau skills and becoming a BI expert. As you progress, you will learn how to build advanced dashboards and improve your storytelling to derive key business insight, as well as make you well-versed with advanced functionalities of Tableau in the business intelligence domain.

Aug 2023 22 hours 48 minutes

Building AI Applications with ChatGPT APIs

Building AI Applications with ChatGPT APIs

This guide covers all ChatGPT API features for effortless creation of robust AI powered apps. With its help, you’ll be able to leverage ChatGPT’s cutting-edge NLP models to take your app development skills to the next level. You’ll also work on ten exciting projects that will give you the practical know-how that you can apply to your existing applications.

Sep 2023 8 hours 36 minutes

Building AI Applications with ChatGPT APIs

Building AI Applications with ChatGPT APIs

This guide covers all ChatGPT API features for effortless creation of robust AI powered apps. With its help, you’ll be able to leverage ChatGPT’s cutting-edge NLP models to take your app development skills to the next level. You’ll also work on ten exciting projects that will give you the practical know-how that you can apply to your existing applications.

Sep 2023 8 hours 36 minutes

Data Engineering with AWS

Data Engineering with AWS

Embark on a journey to master data engineering pipelines on AWS! Our book offers a hands-on experience of AWS services for ingesting, transforming, and consuming data. Whether you're an absolute beginner or someone with basic data engineering experience, this guide is an indispensable resource.

Oct 2023 21 hours 12 minutes

Modern Data Architecture on AWS

Modern Data Architecture on AWS

Every organization wants an agile, performant, and cost-effective data platform that meets all their current and future business needs. Purpose-built AWS analytics services and their features play a big part in building such a modern data platform. This book brings to you all the design and architectural patterns that’ll help you achieve this goal.

Aug 2023 14 hours 0 minutes

Practical Guide to Applied Conformal Prediction in Python

Practical Guide to Applied Conformal Prediction in Python

Discover the power of Conformal Prediction with the "Practical Guide to Applied Conformal Prediction in Python." Master the latest techniques to quantify uncertainty in machine learning and computer vision models, and seamlessly apply them to your industry applications.

Dec 2023 8 hours 0 minutes

TinyML Cookbook

TinyML Cookbook

With over 70 project-based recipes, the TinyML Cookbook is a practical guide that will help you to get the most out of your microcontrollers. It provides a comprehensive understanding of the theoretical foundations while giving you hands-on experience training ML models for deployment on Arduino Nano 33 BLE Sense, Raspberry Pi Pico, and SparkFun RedBoard Artemis Nano microcontrollers.

Nov 2023 22 hours 8 minutes