Recurrent neural networks (RNNs) have become extremely popular for any task that involves sequential data. The core idea behind RNNs is to exploit the sequential information present in the data. Under usual circumstances, every neural network assumes that all of the inputs are independent of each other. However, if we are trying to predict the next word in a sequence or the next point in a time series, it is imperative to use information based on the words used prior or on the historical points in the time series.
One way to perceive the concept of RNNs is that they have a memory that stores information about historical data in a sequence. In theory, RNNs can remember history for arbitrarily long sequences, however, in practice, they do a bad job in tasks where the historical information needs to be retained for more than...
