Sequential workings of RNNs

Recurrent neural networks are a type of artificial neural network designed to recognize and learn patterns in sequences of data. Some of the examples of such sequential data are:

• Handwriting
• Text such as customer reviews, books, source code, and so on
• Spoken word / Natural Language
• Numerical time series / sensor data
• Stock price variation data

Getting ready

In recurrent neural networks, the hidden state from the previous time step is fed back into the network at the next time step, as shown in the following diagram:

Basically, the upward facing arrows going into the network represent the inputs (matrices/vectors) to the RNN at each time step, while the upward-facing arrows coming out of the network...