17.4 Next steps toward machine learning
We can draw a rough boundary between statistical modeling and machine learning. This is a hot topic of debate because — viewed from a suitable distance — all statistical modeling can be described as machine learning.
In this book, we’ve drawn a boundary to distinguish methods based on algorithms that are finite, definite, and effective. For example, the process of using the linear least squares technique to find a function that matches data is generally reproducible with an exact closed-form answer that doesn’t require tuning hyperparameters.
Even within our narrow domain of “statistical modeling,” we can encounter data sets for which linear least squares don’t behave well. One notable assumption of the least squares estimates, for example, is that the independent variables are all known exactly. If the x values are subject to observational error, a more sophisticated approach is required.
The...