Implementing Hyper Band
The extension of Successive Halving, the Hyper Band (HB) method (see Chapter 6), is implemented in the scikit-hyperband
package. This package is built on top of sklearn
, which means it also provides a very similar interface for GridSearchCV
, RandomizedSearchCV
, HalvingGridSearchCV
, and HalvingRandomSearchCV
.
In contrast with the default SH budget definition in the sklearn
implementation, Scikit-Hyperband defines the budget as the number of estimators, n_estimators, in an ensemble of trees, or the number of iterations for estimators trained with stochastic gradient descent, such as the XGBoost algorithm. Additionally, we can use any other hyperparameters that exist in the estimator as the budget definition. However, scikit-hyperband
doesn’t allow us to use the number of samples as the budget definition.
Let’s use the same example as in the Implementing Successive Halving section, but with a different hyperparameter space. Here, we use the...