The first classifier we used, the threshold classifier, was a simple binary classifier. Its result is either one class or the other, as a point is either above the threshold value or it is not. The second classifier we used, the nearest neighbor classifier, was a natural multiclass classifier, its output can be one of the several classes.
It is often simpler to define a simple binary method than the one that works on multiclass problems. However, we can reduce any multiclass problem to a series of binary decisions. This is what we did earlier in the Iris dataset, in a haphazard way: we observed that it was easy to separate one of the initial classes and focused on the other two, reducing the problem to two binary decisions:
Is it an Iris Setosa (yes or no)?
If not, check whether it is an Iris Virginica (yes or no).
Of course, we want to leave this sort of reasoning to the computer. As usual, there are several solutions to this multiclass reduction.
The simplest...