To understand better how we may be able to predict a value using linear regression from first principles, we study a gradient descent algorithm and then implement it in Python.
Gradient descent algorithm and its implementation
Gradient descent algorithm
A gradient descent algorithm is an iterative algorithm updating the variables in the model to fit the data with the least error. More generally, it finds a minimum of a function.
We would like to express the weight in terms of the height using a linear formula:
weight(height,p)=p1*height+p0
We estimate the parameter p=(p0,p1) using n data samples (heighti,weighti) to minimize the following square error:
The gradient descent algorithm does it by updating the parameter pi in...