Parametric models
When it comes to supervised learning, there are two families of learning algorithms: parametric and non-parametric. This area also happens to be a hotbed for gatekeeping and opinion-based conjecture regarding which is better. Basically, parametric models are finite-dimensional, which means that they can learn only a defined number of model parameters. Their learning stage is typically categorized by learning some vector theta, which is also called a coefficient. Finally, the learning function is often a known form, which we will clarify later in this section.
Finite-dimensional models
If we go back to our definition of supervised learning, recall that we need to learn some function, f. A parametric model will summarize the mapping between X, our matrix, and y, our target, within a constrained number of summary points. The number of points is typically related to the number of features in the input data. So, if there are three variables, f will try to summarize the relationship...