Recurrent neural networks
Our thought process always has a sequence. We always understand things in an order. For example, if we watch a movie, we understand the next sequence by connecting it with the previous one. We retain the memory of the last sequence and get an understanding of the whole movie. We don't always go back to the first sequence in order to get it.
Can a neural network act like this? Traditional ones typically cannot operate in this manner and that is a major shortcoming. This is where recurrent neural networks make a difference. It comes with a loop that allows information to flow:
Here, a neural network takes an input as Xt and throws an output in the form of ht . A recurrent neural network is made up of multiple copies of the same network that pass on the message to the successor.
If we were to go and unroll the preceding network, it would look like the following:
This chain-like nature reveals that recurrent neural networks are intimately related to sequences and lists...
