Getting familiar with HTDM
HTDM is designed to help you decide which hyperparameter tuning method should be adopted in a particular situation (see Figure 12.1). Here, the situation is defined based on six aspects:
- Hyperparameter space properties, including the size of the space, types of hyperparameter values (numerical only or mixed), and whether it contains conditional hyperparameters or not
- Objective function complexity: whether it is a cheap or expensive objective function
- Computational resource availability: whether or not you have enough parallel computational resources
- Training data size: whether you have a few, moderate, or a large number of training samples
- Prior knowledge availability: whether you have prior knowledge of the good range of hyperparameter values
- Types of ML algorithms: whether you are working with a small, medium, or large-sized model, and whether you are working with a traditional ML or deep learning type of algorithm
This...