5.5 ESTABLISHING BASELINE MODEL PERFORMANCE
Before evaluating model performance, data scientists should first calibrate the results against some baseline model performance. For example, in the fraud scenario above, suppose we developed a complex fraud detection model that reported 98% accuracy. Sounds impressive, until we remember that an “all negative” model that simply classifies all records as non‐fraudulent would have a 99% accuracy rate. Without comparison to a baseline, our clients cannot determine whether our results are any good.
We offer the following two baseline models for the binary classification case.