Exploring alternative base learners
The base learner is the machine learning model that XGBoost uses to build the first model in its ensemble. The word base is used because it's the model that comes first, and the word learner is used because the model iterates upon itself after learning from the errors.
Decision trees have emerged as the preferred base learners for XGBoost on account of the excellent scores that boosted trees consistently produce. The popularity of decision trees extends beyond XGBoost to other ensemble algorithms such as random forests and extremely randomized trees, which you can preview in the scikit-learn documentation under ExtraTreesClassifier and ExtraTreesRegressor (https://scikit-learn.org/stable/modules/ensemble.html).
In XGBoost, the default base learner, known as gbtree, is one of several base learners. There is also gblinear, a gradient boosted linear model, and dart, a variation of decision trees that includes a dropout technique based on...
