Overfitting
As we have seen in the previous sections, gradient descent and backpropagation are iterative algorithms. One forward and corresponding backward pass through all the training data is called an epoch. With each epoch, the model is trained and the weights are adjusted for minimizing error. In order to test the accuracy of the model, as a common practice, we split the training data into the training set and the validation set.
The training set is used for generating the model that represents a hypothesis based on the historical data that contains the target variable value with respect to the independent or input variables. The validation set is used to test the efficiency of the hypothesis function or the trained model for the new training samples.
Across multiple epochs we typically observe the following pattern:
Figure 4.17: Graph of overfitting model
As we train our neural network through a number of epochs, the loss function error is optimized with every epoch and the cumulative...
