Measuring success in supervised machine learning
As we begin to train our supervised classifiers and regressors, we will need to implement a few ways to determine which models are performing better, thus allowing us to effectively tune the model's parameters and maximize its performance. The best way to achieve this is to understand what success looks like ahead of time before diving into the model development process. There are many different methods for measuring success depending on the situation. For example, accuracy can be a good metric for classifiers, but not regressors. Similarly, a business case for a classifier may not necessarily require accuracy to be the primary metric of interest. It simply depends on the situation at hand. Let's take a look at some of the most common metrics used for each of the fields of classification and regression.
Figure 7.2 – Common success metrics for regression and classification