In the previous sections of the book, we learnt how the response/outcome variable is related to the predictor variables, typically in a supervised learning context. There are various different names for both of those types of variables that people use these days. Let us see some of the synonymous terms for them and we will use them interchangeably in the book:
Input variables (X): Features, predictors, explanatory variables, independent variables
Output variables (Y): Response variable, dependent variable
If there is a relation between Y and X where X=X1, X2, X3,..., Xn (n different predictors) then it can be written as follows:
Here 
is a function that represents how X describes Y and is unknown! This is what we figure out using the observed data points at hand. The term
is a random error term with mean zero and is independent of X.
There are basically two types of errors associated with such an equation - reducible errors and irreducible errors. As the...