For those of us who have spent decades studying machine learning, experience informs the way we choose parameters for our learning algorithms. But for those who are new to it, this is a skill that needs to be developed and this skill comes after learning how learning algorithms work. Once you have finished this book, I believe you will have enough knowledge to choose your parameters wisely. In the meantime, we can discuss some ideas for finding parameters automatically using standard and novel algorithms here.
Before we go any further, we need to make a distinction at this point and define two major sets of parameters that are important in learning algorithms. These are as follows:
- Model parameters: These are parameters that represent the solution that the model represents. For example, in perceptron and linear regression, this would be vector
and scalar 
, while for a deep neural network, this would be a matrix of weights, 
, and a vector of biases, 
. For a convolutional...