How do recurrent neural networks work?

In order to understand how recurrent neural networks work, we need to take a closer look at how recurrent layers in these networks work. There are several different types of recurrent layers you can use in a recurrent neural network. Before we dive into the more advanced versions of recurrent units, let's first discuss how to predict output with a standard recurrent layer, and how to train a neural network that contains recurrent layers.

Making predictions with a recurrent neural network

A basic recurrent layer is quite different from a regular layer in a neural network. Recurrent layers, in general, feature a hidden state that serves as the memory for the layer. There's a loopback...