Gradient Descent
Before we dig into what gradient descent is, let's answer the question of why gradient descent is one of the most important techniques in machine learning. For the sake of simplicity, let's use the linear regression model as an example to illustrate. Let's say we are building a model to predict house prices given the area of the house. The mathematical model is as follows: predicted house price = bias + weight*area of the house.
When we initialize the model, we need to initialize the values for the bias and the weight. Usually, we draw random numbers for the initialization of biases and weights. So, now the model becomes the following: predicted house price = 9 + 5*area of the house.
Bias = 9 and weight = 5 are not ideal model parameters to predict the house price. Currently, the model could be extremely wrong about its house price prediction. It is not even close to being accurate. At the same time, there might exist an ideal value for...
