Designing XGBoost
XGBoost is a significant upgrade from gradient boosting. In this section, you will identify the key features of XGBoost that distinguish it from gradient boosting and other tree ensemble algorithms.
Historical narrative
With the acceleration of big data, the quest to find awesome machine learning algorithms to produce accurate, optimal predictions began. Decision trees produced machine learning models that were too accurate and failed to generalize well to new data. Ensemble methods proved more effective by combining many decision trees via bagging and boosting. A leading algorithm that emerged from the tree ensemble trajectory was gradient boosting.
The consistency, power, and outstanding results of gradient boosting convinced Tianqi Chen from the University of Washington to enhance its capabilities. He called the new algorithm XGBoost, short for Extreme Gradient Boosting. Chen's new form of gradient boosting included built-in regularization and impressive...
