In the previous chapter, we studied CART as a powerful recursive partitioning method, useful for building (nonlinear) models. Despite the overall generality, CART does have certain limitations that necessitate some enhancements. It is these extensions that form the crux of the final chapter of the book. For technical reasons, we will focus solely on the classification trees in this chapter. We will also briefly look at some limitations of the CART tool.
One improvement of the CART is provided by the bagging technique. In this technique, we build multiple trees on the bootstrap samples drawn from the actual dataset. An observation is put through each of the trees and a prediction is made for its class, and, based on the majority prediction of its class, it is predicted to belong to the majority count class. A different approach is provided by random forests, where one compares a random pool of covariates against the observations. We finally consider another important...