Introducing RNNs
To address the limitations of FNNs and CNNs regarding sequential data, we need a network that can meet 2 requirements.
- It can takes sequences of non-fixed lengths, one element of the sequence at a time.
- It must not only identify the nonlocal relationships in the sequence but also remember the most important events that happened before.
This idea led to the development of RNNs, which are a variant of DNN with a feedback loop (hidden state) that can feed the results back into the network and make them part of the final output (Figure 6.3):
Figure 6.3 – Recurrent neural network
RNNs capture previous observations or historical events up to the current timestamp and because the hidden state of the current stamp is the same as the previous timestamp, the computation is recurrent (hence why they are referred to as RNNs):
Figure 6.4 – Difference between a standard neural network (a) and a recurrent...
