Tuning the models

One common problem in model development is overfitting. Overfitting happens when the model performs well on the training data but does not generalize well on unseen data. There are several reasons for overfitting, such as high model complexity, training for many epochs, too little training data, and so on. Model tuning is the processing of increasing model performance by limiting model complexity, regularization, dropout, and so on to reduce overfitting. This is generally done in DL by optimizing “hyperparameters”.

Before we further discuss tuning models, let’s understand the difference between parameters and hyperparameters. Parameters are inputs to the ML library or model that can be generally learned from the model. Some examples of the parameters of NNs include weights and biases. During model training, through backpropagation, the model learns those parameters, whereas hyperparameters are those parameters that cannot be learned from the...