"The true logic of this world is the calculus of probabilities."
- James Clerk Maxwell, Scottish physicist
In the previous chapter, we took a look at using Ordinary Least Squares (OLS) to predict a quantitative outcome or, in other words, linear regression. It's now time to shift gears somewhat and examine how we can develop algorithms to predict qualitative outcomes. Such outcome variables could be binary (male versus female, purchase versus doesn't purchase, or a tumor is benign versus malignant) or multinomial categories (education level or eye color). Regardless of whether the outcome of interest is binary or multinomial, our task is to predict the probability of an observation belonging to a particular category of the outcome variable. In other words, we develop an algorithm to classify the observations.
To begin exploring classification problems, we'll discuss why applying the OLS linear regression isn't the correct technique and how the algorithms introduced...