Recurrent Neural Networks (RNNs) are a special type of neural network designed to work with sequential data. They are popular for time series forecasting as well as for solving NLP problems such as machine translation, text generation, and speech recognition. There are numerous extensions of the RNNs, such as Long-Short Term Memory (LSTM) networks and Gated Recurrent Unit (GRU) networks, which are currently part of some of the state-of-the-art architectures. However, it is good to be familiar with the original vanilla RNN. The following diagram presents the typical RNN schema:
One of the main differences between the feedforward networks and RNNs is that the former take a fixed size input at once to produce a fixed size output. On the other hand, RNNs do not take all the input data at once – they ingest the data sequentially...
