Tuning your prediction model is vital for getting the best possible output for your data mining work. There are two types of parameters introduced in this chapter. The first are internal parameters of the hypothesis function, and are stored as individual θ's in the weights vector Θ. These parameters are tuned during the minimization of the loss function. The second type are constants added to the loss function or the minimization (for example, gradient descent) function that influences the tuning of the internal parameters, and are called hyperparameters. The hyperparameters are the subject of the tuning strategies in this section.
Hyperparameter tuning is often referred to as tuning the knobs by practitioners in the field. This is a call-back to the analog days of engineering, when analytical machines had actual physical knobs. Back...