Introduction

Sequential data refers to datasets in which each data point is dependent on the previous ones. Think of it like a sentence, which is composed of a sequence of words that are related to each other. A verb will be linked to a subject and an adverb will be related to a verb. Another example is a stock price, where the price on a particular day is related to the price of the previous days. Traditional neural networks are not fit for processing this kind of data. There is a specific type of architecture that can ingest sequences of data. This chapter will introduce you to such models—known as recurrent neural networks (RNNs).

An RNN model is a specific type of deep learning architecture in which the output of the model feeds back into the input. Models of this kind have their own challenges (known as vanishing and exploding gradients) that will be addressed later in the chapter.

In many ways, an RNN is a representation of how a brain might work. RNNs use...