In previous chapters, we've considered how to describe samples in terms of summary statistics and how population parameters can be inferred from them. Such analysis tells us something about a population in general and a sample in particular, but it doesn't allow us to make very precise statements about individual elements. This is because so much information has been lost by reducing the data to just two statistics: the mean and standard deviation.

We often want to go further and establish a relationship between two or more variables or to predict one variable given another. This takes us into the study of correlation and regression. Correlation concerns the strength and direction of the relationship between two or more variables. Regression determines the nature of this relationship and enables us to make predictions from it.

Linear regression is our first machine learning algorithm. Given a sample...