Introducing boosting algorithms
The term boosting refers to a family of algorithms that use ensemble learning to build a collectively robust classifier from several weak classifiers. The difference with other ensemble techniques is that in boosting, we build a series of trees, where every other tree tries to fix the mistakes made by its predecessor. Contrast this approach with how the random forest classifier performs decisions presented in the Contracting a decision tree section of Chapter 3, Classifying Topics of Newsgroup Posts. In that case, multiple trees are constructed in parallel using the bagging technique. Another distinctive characteristic of boosting algorithms is their ability to deal with the bias-variance trade-off discussed in the Applying regularization section of Chapter 4, Extracting Sentiments from Product Reviews. Let’s present the major boosting algorithms in the following sections.
Understanding AdaBoost
Adaptive Boosting (AdaBoost) was the first...