Hyperparameter optimization
A SageMaker Automatic Model Tuning job allows you to run multiple training jobs with a unique combination of hyperparameters in parallel. In other words, a single tuning job creates multiple SageMaker training jobs. Hyperparameter tuning allows you to speed up your model development and optimization by trying many combinations of hyperparameters in parallel and iteratively moving toward more optimal combinations. However, it doesn’t guarantee that your model performance will always improve. For instance, if the chosen model architecture is not optimal for the task at hand or your dataset is too small for the chosen model, you are unlikely to see any improvements when running hyperparameter optimizations.
When designing for your tuning job, you need to consider several key parameters of your tuning job, as follows:
- Search algorithm (or strategy): This defines how SageMaker chooses the next combination of hyperparameters.
- Hyperparameters...