There have been many extensions of vanilla RNNs over the past several years. This is by no means an exhaustive list of all of the great advances in RNNs that are happening in the community, but we're going to review a couple of the most notable ones: Bidirectional RNNs, and NTM.
Extensions of RNNs
Bidirectional RNNs
In recent years, researchers have developed several improvements on the traditional RNN structure. Bidirectional RNNs were developed with the idea that they may not only depend on the information that came before in a sequence, but also the information that comes afterwards. Structurally, they are just two RNNs that are stacked on top of each other, and their output is a combination of the hidden states of...
