Chapter 4
The Bias-Variance Trade-off
Section 6
Cross Validation: Choosing the Regularization Parameter and Other Hyperparameters
By now, you should be interested in using regularization in order to decrease the overfitting we observed when we tried to model the synthetic data. The question is, how do we choose the regularization parameter, C? C is an example of a model hyperparameter. Hyperparameters are different from the parameters that are estimated when a model is trained, such as the coefficients and the intercept of a logistic regression. Rather than being estimated by an automated procedure like the parameters are, hyperparameters are input directly by the user as keyword arguments, typically when instantiating the model class. So, how do we know what values to choose? Here are the topics that we will cover now: