Implementing Bayesian Optimization Gaussian Process
Bayesian Optimization Gaussian Process (BOGP) is one of the variants of the Bayesian Optimization hyperparameter tuning group (see Chapter 4, Exploring Bayesian Optimization). To implement BOGP, we can utilize the skopt
package. Similar to scikit-hyperband
, this package is also built on top of the sklearn
package, which means the interface for the implemented Bayesian Optimization tuning class, BayesSearchCV
, is very similar to GridSearchCV
, RandomizedSearchCV
, HalvingGridSearchCV
, HalvingRandomSearchCV
, and HyperbandSearchCV
.
However, unlike sklearn
or scikit-hyperband
, which works well directly with the distribution implemented in scipy
, in skopt
, we can only use the wrapper provided by the package when defining the hyperparameter space. The wrappers are defined within the skopt.space.Dimension
instances and consist of three types of dimensions, such as Real
, Integer
, and Categorical
. Within each of these dimension wrappers...