Understanding BOHB
Bayesian Optimization and Hyper Band (BOHB) is an extension of HB that is superior to CFS, SH, and HB, in terms of understanding the relationship between the hyperparameter candidates and the objective function. If CFS, SH, and HB are all part of the informed search group based on random search, BOHB is an informed search group that is based on the BO method. This means BOHB is able to decide which subspace needs to be searched based on previous experiences rather than luck.
As its name implies, BOHB is the combination of the BO and HB methods. While SH and HB can also be utilized with other black-box methods (see the Understanding SH and Understanding HB sections), BOHB is specifically designed to utilize a BO method in a way that can support HB. Furthermore, the BO method in BOHB also tracks all the previous evaluations on all budgets, so that it can serve as the base for future evaluations. Note that the BO method used in BOHB is the multivariate TPE, which...