Gradient descent algorithm and its implementation

To understand how we may be able to predict a value by using linear regression from first principles in an even better way, we need to study the gradient descent algorithm and then implement it in Python.

Gradient descent algorithm

A gradient descent algorithm is an iterative algorithm that updates the variables in the model to fit the data, making as few errors as possible. More generally, it finds the minimum of a function.

We would like to express the weight in terms of height by using a linear formula:

We estimate the parameter,  , using n data samples  to minimize the following square error:

The gradient descent algorithm does this by updating the...