Chapter 8, Neural Networks and Deep Learning, included an example of a neural network built from scratch in Go. This neural network included an implementation of the backpropagation method to train neural networks, which can be found in almost any neural network code. We discussed some details in that chapter. However, this method is utilized so often that we wanted to go through it step by step here.
To train a neural network with backpropagation, we do the following for each of a series of epochs:
- Feed the training data through the neural network to produce output.
- Calculate an error between the expected output and the predicted output.
- Based on the error, calculate updates for the neural network weights and biases.
- Propagate these updates back into the network.
As a reminder, our implementation of this procedure for a network with a single hidden layer looked...
