We have been talking about training the model. What does that mean in practice?
In supervised learning, the dataset is usually split into three non-equal parts: training, validation, and test:
- The training set on which you train your model. It has to be big enough to give the model as much information on the data as possible. This subset of the data is used by the algorithm to estimate the best parameters of the model. In our case, the SGD algorithm will use that training subset to find the optimal weights of the linear regression model.
- The validation set is used to assess the performance of a trained model. By measuring the performance of the trained model on a subset that has not been used in its training, we have an objective assessment of its performance. That way we can train different models with different meta parameters and see which one is performing the best on the validation set. This is also called model selection. Note that this creates a feedback...