In the previous chapter, we took a look at using Ordinary Least Squares (OLS) to predict a quantitative outcome, in other words, linear regression. It is now time to shift gears somewhat and examine how we can develop algorithms to predict qualitative outcomes. Such outcome variables could be binary (male versus female, purchases versus does not purchase, tumor is benign versus malignant) or multinomial categories (education level or eye color). Regardless of whether or not the outcome of interest is binary or multinomial, the task of the analyst is to predict the probability that an observation would belong to which category of the outcome variable. In other words, we develop an algorithm in order to classify the observations.

To begin exploring the classification problems, we will discuss why applying the OLS linear regression...