-
Book Overview & Buying
-
Table Of Contents
The Kaggle Book - Second Edition
By :
In this chapter, we discussed hyperparameter optimization at length as a way to increase your model’s performance and score higher on the leaderboard. We started by explaining the code functionalities of scikit-learn, such as grid search and random search, as well as the newer halving algorithms.
Then, we progressed to Bayesian optimization and explored scikit-optimize, KerasTuner, and, finally, Optuna. We spent more time discussing the direct modeling of the surrogate function by GPs and how to hack it because it can allow you greater intuition and a more ad hoc solution.
We recognize that, at the moment, Optuna has become a gold standard among Kagglers for tabular competitions and deep neural network ones because of its speedier convergence to optimal parameters in the time allowed in a Kaggle Notebook. However, if you want to stand out among the competition, you should strive to test solutions from other optimizers as well.
In the next chapter, we will...