Implementing Backpropagation

You can build a neural network by combining the layers implemented in the previous sections as if you were assembling Lego blocks. Here, we will build a neural network by combining the layers we've implemented so far.

Overall View of Neural Network Training

Because my description was a little long, let's check the overall view of neural network training again before proceeding with its implementation. Now we will take a look at the procedure for neural network training.

Presupposition

A neural network has adaptable weights and biases. Adjusting them so that they fit the training data is called "training." Neural network training consists of the following four steps:

Step 1 (mini-batch):

Select some data at random from the training data.

Step 2 (calculating the gradients):

Obtain the gradient of the loss function for each weight parameter.

Step 3 (updating the parameters):

Update the parameters slightly...