Recurrent Neural Networks (RNNs) are the most flexible form of networks and are widely used in natural language processing (NLP), financial services, and a variety of other fields. Vanilla feedforward networks, as well as their convolutional varieties, accept a fixed input vector and output a fixed vector; they assume that all of your input data is independent of each other. RNNs, on the other hand, operate on sequences of vectors and output sequences of vectors, and allow us to handle many exciting types of data. RNNs are actually turing-complete, in that they can simulate arbitrary tasks, and hence are very appealing models from the perspective of the Artificial Intelligence scientist.
In this chapter, we'll introduce the core building blocks of RNNs, including special RNN units called Gated Recurrent Units (GRU) and Long short-term memory units...
