Varying the learning rate to improve network accuracy

So far, in the previous recipes, we used the default learning rate of the Adam optimizer, which is 0.0001.

In this section, we will manually set the learning rate to a higher number and see the impact of changing the learning rate on model accuracy, while reusing the same MNIST training and test dataset that were scaled in the previous recipes.

Getting ready

In the previous chapter on building feedforward neural networks, we learned that the learning rate is used in updating weights and the change in weight is proportional to the amount of loss reduction.

Additionally, a change in a weight's value is equal to the decrease in loss multiplied by the learning rate. Hence...