Regularization

Overfitting often creates difficulties in machine learning problems. In overfitting, the model fits the training data too well and cannot properly handle other data that is not contained in the training data. Machine learning aims at generalizing performance. It is desirable for the model to properly recognize unknown data that is not contained in the training data. While you can create a complicated and representative model this way, reducing overfitting is also important:

Figure 6.18: Effect of batch norm – batch norm accelerates learning

Overfitting

The main two causes of overfitting are as follows:

• The model has many parameters and is representative.
• The training data is insufficient.

Here, we will generate overfitting by providing these two causes. Out of 60,000 pieces of training data in the MNIST dataset, only 300 are provided, and a seven-layer network is used to increase the network's complexity. It...