Introducing sequential data

Let's begin our discussion of RNNs by looking at the nature of sequential data, more commonly known as sequences. We'll take a look at the unique properties of sequences that make them different from other kinds of data. We'll then see how we can represent sequential data, and explore the various categories of models for sequential data, which are based on the input and output of a model. This will help us explore the relationship between RNNs and sequences a little bit later on in the chapter.

Modeling sequential data – order matters

What makes sequences unique, from other data types, is that elements in a sequence appear in a certain order, and are not independent of each other.

If you recall from Chapter 6, Learning Best Practices for Model Evaluation and Hyperparameter Tuning, we discussed that typical machine learning algorithms for supervised learning assume that the input data is Independent and Identically Distributed (IID). For example, if we have n data...