Hyperparameter tuning in Python
Let's get a taste of what hyperparameter tuning looks like in practice. We will use one of the most popular Python hyperparameter optimization libraries called Hyperopt. First, let's get a general idea of how to use it in practice.
Hyperopt is a Python library that provides an easy-to-use API that requires the following three objects:
- A search space
- An objective function
- An optimization method
Let's look at each of these requirements in detail:
- Search space is simply the space within which the optimizer will search for different hyperparameter options. The library provides the following parameter expressions:
- Categorical parameters – Parameter values that are purely categorical and can even be non-scalar (they do not need to be a number). They are provided via the
hyperopt.hp.choice
method. - Integer parameters – Integer value parameters obtained via methods such as
hyperopt.hp.quniform
orhyperopt...
- Categorical parameters – Parameter values that are purely categorical and can even be non-scalar (they do not need to be a number). They are provided via the