Regularizing with dropout

In this recipe, we will add dropout to a GRU to add regularization to the IMDb classification dataset.

Getting ready

Just like fully connected neural networks, recurrent neural networks such as GRUs and LSTMs can be trained with dropout. As a reminder, dropout is just randomly setting some unit’s activation to zero during training. As a result, it allows a network to have less information at once and to hopefully generalize better.

We will improve upon the results of the GRU training recipe, by using dropout on the same task – the IMDb dataset binary classification.

If not already done, the dataset can be downloaded using the Kaggle API with the following command line:

kaggle datasets download -d lakshmi25npathi/imdb-dataset-of-50k-moviereviews --unzip

The required libraries can be installed with the following:

pip install pandas numpy scikit-learn matplotlib torch transformers

How to do it…

Here are the steps...