So far, we have learned a lot about predictive analytics and the fundamentals of regression models—classification models—including simple models such as multiple linear regression and very complex models such as multilayer perceptrons. We know how to train models to make predictions and that it is very important to set apart a testing set for evaluation because we want to evaluate how the model will perform with data that it has not seen before, that is, we want the model to learn something that can be generalized to unseen data.
So far, we have been using generic metrics to evaluate model performance—Mean Squared Error (MSE) for regression problems and accuracy for classification problems. However, in every predictive analytics project, you have to think carefully about the metrics you are using for evaluating the model and the general evaluation...