Implementing Adaptive TPE
Adaptive TPE (ATPE) is a variant of the TPE hyperparameter tuning method that is developed based on several improvements compared to TPE, such as automatically tuning several hyperparameters of the TPE method based on the data that we have. For more information about this method, please refer to the original white papers. These can be found in the GitHub repository of the author (https://github.com/electricbrainio/hypermax).
While you can experiment with this method directly using the original GitHub repository of ATPE, Hyperopt
has also included this method as part of the package. You can simply follow a similar procedure as in the Implementing Random Search section by only changing the algo
parameter to atpe.suggest
in Step 4. The following code shows how to perform hyperparameter tuning with ATPE in Hyperopt
. Please note that ATPE utilizes the LightGBM model to predict each of the ATPE parameters. That’s why we need to have the LightGBM
package...