Recurrent neural networks are great for tasks involving sequential data. However, they do come with their drawbacks. This section will highlight and discuss one such drawback, known as the vanishing gradient problem.
Pain point #1 – The vanishing gradient problem
Getting ready
The name vanishing gradient problem stems from the fact that, during the backpropagation step, some of the gradients vanish or become zero. Technically, this means that there is no error term being propagated backward during the backward pass of the network. This becomes a problem when the network gets deeper and more complex.
