Implementing Bayesian Optimization Random Forest
Bayesian Optimization Random Forest (BORF) is another variant of Bayesian Optimization hyperparameter tuning methods that utilize RF as the surrogate model. Note that this variant is different from Sequential Model Algorithm Configuration (SMAC) although both of them utilize RF as the surrogate model (see Chapter 4, Exploring Bayesian Optimization).
Implementing BORF with skopt
is actually very similar to implementing BOGP as discussed in the previous section. We just need to change the base_estimator
parameter within optimizer_kwargs
to RF. Let’s use the same example as in the Implementing Bayesian Optimization Gaussian Process section, but change the acquisition function from EI to LCB. Additionally, let’s change the xi
parameter in the acq_func_kwargs
to kappa since we are using LCB as our acquisition function. Note that we can also still use the same acquisition function. The changes made here just to show how you...