Understanding hyper band
Hyper Band (HB) is an extension of SH that is specifically designed to overcome issues inherent in SH (see Figure 6.7). Although we can perform meta-learning to help us balance the trade-off, most of the time we do not have the metadata that’s needed in practice. Furthermore, the possibility of SH removing better sets of hyperparameters in the first several iterations is also worrying and can’t be solved by just finding a sweet spot from the trade-off. HB tries to solve these issues by calling SH several times iteratively.
Since HB is just an extension of SH, it is suggested that you utilize HB as your hyperparameter tuning method when you are working with a large model (for example, a deep neural network) and/or working with a large amount of data, just like SH. Furthermore, it is even better to utilize HB than SH when you do not have the time or metadata needed to help you configure the trade-off between the amount of resources and the number...