EXERCISES
CLARIFYING THE CONCEPTS
- Explain the difference between model evaluation and model validation.
- What does a contingency table consist of?
- Show that GT = TPN + TPP and that TAP = GT − TAN.
- Show that Error Rate = 1 − Accuracy.
- What do sensitivity and specificity measure?
- Explain the Method for Model Evaluation.
- Why did we choose the All Negative model as a baseline to calibrate the accuracy of Model 1, rather than the All Positive model?
- When error costs are unequal, what is the most important evaluation measure for the purpose of model selection?
- Explain how a naïve analyst would erroneously prefer Model 1 to Model 2.
- For the All Positive and All Negative models, calculate the evaluation metrics from Table 7.11.
WORKING WITH THE DATA
For Exercises 11–22, work with the clothing_data_driven_training and clothing_data_driven_test data sets. Use R to solve each problem.
- Using the training data set, create a...