The third method of classification we'll look at in this chapter is the decision tree. A decision tree models the process of classification as a series of tests that checks the value of a particular attribute or attributes of the item to be classified. It can be thought of as similar to a flowchart, with each test being a branch in the flow. The process continues, testing and branching, until a leaf node is reached. The leaf node will represent the most likely class for the item.

Decision trees share some similarities with both logistic regression and naive Bayes. Although the classifier can support categorical variables without dummy coding, it is also able to model complex dependencies between variables through repeated branching.

In the old-fashioned parlor game Twenty Questions, one person, the "answerer", chooses an object but does not reveal their choice to the others. All other players are "questioners" and take turns to ask questions that aim to guess the object the...