We now move on to one of the easily interpretable and most popular classifiers there are out there: the decision tree. Decision trees, which look like an upside down tree with the trunk on top and the leaves on the bottom, play an important role in situations where classification decisions have to be transparent and easily understood and explained. They also handle both continuous and categorical predictors, outliers, and irrelevant predictors rather gracefully. Finally, the general idea behind the algorithms that create decision trees are quite intuitive, though the details can sometimes get hairy.
Figure 10.7 depicts a simple decision tree designed to classify motor vehicles into either motorcycles, golf carts, or sedans:
Figure 10.7: A simple and illustrative decision tree that classifies motor vehicles into either motorcycles, golf carts, or sedans
This is a rather simple decision tree with only three leaves (terminal nodes) and two decision points. Note that the first decision...