In this chapter, we're going to discuss binary decision trees and ensemble methods. Even if they're probably not the most common methods for classification, they offer a good level of simplicity and can be adopted in many tasks that don't require a high level of complexity. They're also quite useful when it's necessary to show how a decision process works because they are based on a structure that can be shown easily in presentations and described step by step.

Ensemble methods are a powerful alternative to complex algorithms because they try to exploit the statistical concept of majority vote. Many weak learners can be trained to capture different elements and make their own predictions, which are not globally optimal, but using a sufficient number of elements, it's statistically probable that a majority will evaluate correctly. In particular, we're going to discuss random forests of decision trees and some boosting methods that are slightly...