Records and supervised learning
For the purpose of this chapter, a record is an observation or measurement of one or several attributes. We assume that the observations might contain noise (or be inaccurate for one or other reason):
While we believe that there is some pattern or correlation between the attributes, the one that we are after and want to uncover, the noise is uncorrelated across either the attributes or the records. In statistical terms, we say that the values for each record are drawn from the same distribution and are independent (or i.i.d. in statistical terms). The order of records does not matter. One of the attributes, usually the first, might be designated to be the label.
Supervised learning is when the goal is to predict the label yi:
Here, N is the number of remaining attributes. In other words, the goal is to generalize the patterns so that we can predict the label by just knowing the other attributes, whether because we cannot physically get the measurement or just...