Implementing Hyperband
Implementing Hyperband (HB) in Optuna
is very similar to implementing Successive Halving as a pruner. The only difference is that we have to set the pruner
parameter in the optimize()
method to optuna.pruners.HyperbandPruner()
in step 2 in the preceding section. The following code shows you how to perform hyperparameter tuning with the Random Search algorithm as the sampler and HB as the pruner:
study = optuna.create_study(direction='maximize',
sampler=optuna.samplers.RandomSampler(seed=0),
pruner=optuna.pruners.HyperbandPruner(reduction_factor=3, min_resource=5)
)
All of the parameters of HyperbandPruner
are the same as SuccessiveHalvingPruner
’s, except that, here, there is no min_early_stopping_rate
parameter and there is a max_resource
parameter...