Exploring alternative base learners
The base learner is the machine learning model that XGBoost uses to build the first model in its ensemble. The word base is used because it's the model that comes first, and the word learner is used because the model iterates upon itself after learning from the errors.
Decision trees have emerged as the preferred base learners for XGBoost on account of the excellent scores that boosted trees consistently produce. The popularity of decision trees extends beyond XGBoost to other ensemble algorithms such as random forests and extremely randomized trees, which you can preview in the scikit-learn documentation under
In XGBoost, the default base learner, known as
gbtree, is one of several base learners. There is also
gblinear, a gradient boosted linear model, and
dart, a variation of decision trees that includes a dropout technique based on...