In Chapter 3, Logistic Regression and Discriminant Analysis we discussed using logistic regression to determine the probability that a predicted observation belongs to a categorical response—what we refer to as a classification problem. Logistic regression was just the beginning of classification methods, with a number of techniques that we can use to improve our predictions.

In this chapter, we will delve into two nonlinear techniques: K-Nearest Neighbors (KNN) and Support Vector Machines (SVM). These techniques are more sophisticated than what we've discussed earlier because the assumptions on linearity can be relaxed, which means a linear combination of the features in order to define the decision boundary is not needed. Be forewarned though, this does not always equal superior...