Making a Network Deeper

Throughout this book, we have learned a lot about neural networks, including the various layers that constitute a neural network, effective techniques used in training, CNNs that are especially effective for handling images, and how to optimize parameters. These are all important techniques in deep learning. Here, we will integrate the techniques we have learned so far to create a deep network. Then, we will try our hand at handwritten digit recognition using the MNIST dataset.

Deeper Networks

First, we will create a CNN that has the network architecture shown in Figure 8.1. This network is based on the VGG network, which will be described in the next section.

As shown in Figure 8.1, the network is deeper than the networks that we have implemented so far. All the convolution layers used here are small 3x3 filters. Here, the number of channels becomes larger as the network deepens (as the number of channels in a convolution layer increases from 16 in...