Implementing TPE
TPE is one of the variants of the Bayesian optimization hyperparameter tuning group (see Chapter 4), which is the default sampler in Optuna
. To perform hyperparameter tuning with TPE in Optuna
, we can just simply pass the optuna.samplers.TPESampler()
class to the sampler parameter of the create_study()
function. The following example shows how to implement TPE in Optuna
. We’ll use the same data as in the examples in Chapter 7 and follow the steps introduced in the preceding section as follows:
- Define the
objective
function along with the hyperparameter space. Here, we’ll use the same function that we defined in the Introducing Optuna section. Remember that we use the train-validation split instead of the k-fold cross-validation method within theobjective
function. - Initiate a
study
object via thecreate_study()
function as follows:study = optuna.create_study(direction='maximize', sampler=optuna.samplers.TPESampler(seed=0))
- Perform...