-
Book Overview & Buying
-
Table Of Contents
The Kaggle Book - Second Edition
By :
Leaving behind grid search (feasible only when the space of experiments is limited), the usual choice for the practitioner is to apply random search optimization or try a Bayesian optimization (BO) technique, which requires a more complex setup.
Originally introduced in the paper Practical Bayesian optimization of machine learning algorithms by Snoek, J., Larochelle, H., and Adams, R. P. (http://export.arxiv.org/pdf/1206.2944), the key idea behind Bayesian optimization is that we optimize a proxy function, also called a surrogate function, rather than directly relying on the true objective function, something that grid search and random search both do.
In other words, a proxy function is a simpler or computationally cheaper model that approximates the true objective function. It allows us to make informed decisions about where to sample next in the parameter space without having to evaluate the actual objective function, which can be costly or time-consuming...