-
Book Overview & Buying
-
Table Of Contents
Machine Learning For Dummies
By :
Seeing a large difference between the cross-validation (CV) estimates and the result is a common problem that appears with a test set or fresh data. Having this problem means that something went wrong with the cross-validation. Beyond the fact that CV isn’t a good performance predictor, this problem also means that a misleading indicator has induced you to model the problem incorrectly and achieve unsatisfactory results.
Cross-validation provides you with hints when the steps you take (data preparation, data and feature selection, hyper-parameter fixing, or model selection) are correct. It’s important, but not critical, that CV estimates precisely replicate out-of-sample error measurements. However, it is crucial that CV estimates correctly reflect improvement or worsening in the test phase due to your modeling decisions. Generally, there are two reasons that the cross-validation estimates can vary from the true error results:
Snooping...
Change the font size
Change margin width
Change background colour