Another popular method for regularization is dropout. A dropout forces a neural network to learn multiple independent representations by randomly removing connections between neurons in the learning phase. For example, when using a dropout of 0.5, the network has to see each example twice before the connection is learned. Therefore, a network with dropout can be seen as an ensemble of networks.Â
In the following recipe, we will improve a model that clearly overfits the training data by adding dropouts.