Introducing Optuna
Optuna
is a hyperparameter tuning package in Python that provides several hyperparameter tuning methods implementation, such as Grid Search, Random Search, Tree-structured Parzen Estimators (TPE), and many more. Unlike Hyperopt
, which assumes we are always working with a minimization problem (see Chapter 8, Hyperparameter Tuning via Hyperopt), we can tell Optuna
the type of optimization problem we are working on: minimization or maximization.
Optuna
has two main classes, namely samplers and pruners. Samplers are responsible for performing the hyperparameter tuning optimization, whereas pruners are responsible for judging whether we should prune the trials based on the reported values. In other words, pruners act like early stopping methods where we will stop a hyperparameter tuning iteration whenever it seems that there’s no additional benefit to continuing the process.
The built-in implementation for samplers includes several hyperparameter tuning...