This quote from Fernández-Delgado et al. in the Journal of Machine Learning Research is meant to set the stage that the techniques in this chapter are quite powerful, particularly when used for classification problems. Certainly, they are not always the best solution but they do provide a good starting point.

In the previous chapters, we examined the techniques to predict either a quantity or a label classification. Here we will apply them on both types of problems. We will also approach the business problem differently than in the previous chapters. Instead of defining a new problem, we will apply the techniques to some of the issues that we already tackled, with an eye to see if we can improve our predictive power. For all intents and purposes, the business case in this chapter is to see if we can improve on the models that we selected before.

The first item of discussion is the basic decision tree, which is both simple to build and to understand. However, the single decision...