Recurrent neural networks
The models that we have studied till now respond only present input. You present them an input, and based on what they have learned, they give you a corresponding output. But this is not the way we humans work. When you are reading a sentence, you do not interpret each word individually, you take the previous words into account to conclude its semantic meaning.
RNNs are able to address this issue. They use the feedback loops, which preserves the information. The feedback loop allows the information to be passed from the previous steps to the present. The following diagram shows the basic architecture of an RNN and how the feedback allows the passing of information from one step of the network to the next (Unroll):
Recurrent neural network
In the preceding diagram, X represents the inputs. It's connected to the neurons in the hidden layer by weights Whx, the output of the hidden layer, h, is fed back to the hidden layer via weights Whh, and also contributes to the output...
