Hyperparameter could be considered as high-level parameter which determines one of the various properties of a model such as complexity, training behavior and learning rate. These parameters naturally differ from model parameters as they need to be set before training starts.
For example, the k in k-means or k-nearest-neighbors is a hyperparameter for these algorithms. The k in k-means denotes the number of clusters to be found, and the k in k-nearest-neighbors denotes the number of closest records to be used to make predictions.
Tuning hyperparameters is a crucial step in any machine learning project to improve predictive performance. There are different techniques for tuning, such as grid search, randomized search and bayesian optimization, but these techniques are beyond the scope of this chapter.
Let's have a quick look at the k-means algorithms parameters...