7.1 INTRODUCTION TO MODEL EVALUATION
So far in Data Science Using Python and R, we have covered the first five phases of the Data Science Methodology:
- Data Understanding Phase
- Data Preparation Phase
- Exploratory Data Analysis Phase
- Setup Phase
- Modeling Phase (at least a little bit)
But, so far we have not examined whether our models are any good. That is, we have not evaluated their usefulness in making predictions. Note the difference between evaluation and validation. Model validation simply makes sure that our model results are consistent between the training and test data sets. But, model validation does not tell us how accurate our models are, or what their error rate is. For measures like these, we need to turn to model evaluation. Since the only models we have learned so far are decision trees for classification, we shall restrict our discussion to evaluative measures for classification models.