Understanding the three datasets
In reality, you don't need one dataset, but ideally three. These are required for training, validation, and testing. Before defining them, please consider that unfortunately sometimes, there is some confusion regarding the meaning of validation and test, typically where only two datasets are available, as in this case, validation and test datasets coincide. We did the same in Chapter 4, Deep Learning with Neural Networks, where we used the test dataset as validation.
Let's now define these three datasets, and then we can explain how ideally we should have tested the MNIST dataset:
- Training dataset: This is the dataset used to train the neural network, and it is typically the biggest of the three datasets.
- Validation dataset: This is usually a hold-out part of the training dataset that is not used for training, but only to evaluate the performance of a model and tune its hyperparameters (for example, the topology of the network...
