The rules of probability
At the simplest level, a model, be it for machine learning or a more classical method such as linear regression, is a mathematical description of how various kinds of data relate to one another.
In the task of modeling, we usually think about separating the variables of our dataset into two broad classes:
- Independent data, which primarily means inputs to a model, are denoted by X. These could be categorical features (such as a "0" or "1" in six columns indicating which of six schools a student attends), continuous (such as the heights or test scores of the same students), or ordinal (the rank of a student in the class).
- Dependent data, conversely, are the outputs of our models, and are denoted by Y. (Note that in some cases Y is a label that can be used to condition a generative output, such as in a conditional GAN.) As with the independent variables, these can be continuous, categorical, or ordinal, and they can...