Bagging is essentially an ensembling method that consists of homogeneous base learners. Bagging was introduced as a bootstrap aggregation method, and we saw some of the advantages of the bootstrap method in Chapter 2, Bootstrapping. The advantage of the bagging method is the stabilization of the predictions. This chapter began with modifications for the classification tree, and we saw different methods of improvising the performance of a decision tree so that the tree does not overfit the data. The bagging of the decision tress and the related tricks followed in the next section. We then introduced k-NN as an important classifier and illustrated it with a simple example. The chapter concluded with the bagging extension of the k-NN classifier.

Bagging helps in reducing the variance of the decision trees. However, the trees of the two bootstrap samples are correlated since a lot of common observations generate them. In the next chapter, we will look at innovative resampling, which will...