Implementing simulated annealing
Simulated annealing (SA) is part of the Heuristic Search hyperparameter tuning group (see Chapter 5, Exploring Heuristic Search), which is also implemented in the Hyperopt
package. Similar to TPE and ATPE, to perform hyperparameter tuning with this method, we can simply follow the procedure shown in the Implementing Random Search section; we only need to change the algo
parameter to anneal.suggest
in Step 4. The following code shows how to perform hyperparameter tuning with SA in Hyperopt
:
from hyperopt import fmin, anneal best = fmin(objective, space=hyperparameter_space, algo=anneal.suggest, max_evals=100, rstate=np.random.default_rng(0), ...