Modeling Sequential Data Using Recurrent Neural Networks

In the previous chapter, we focused on convolutional neural networks (CNNs). We covered the building blocks of CNN architectures and how to implement deep CNNs in TensorFlow. Finally, you learned how to use CNNs for image classification. In this chapter, we will explore recurrent neural networks (RNNs) and see their application in modeling sequential data.

We will cover the following topics:

• Introducing sequential data
• RNNs for modeling sequences
• Long short-term memory (LSTM)
• Truncated backpropagation through time (TBPTT)
• Implementing a multilayer RNN for sequence modeling in TensorFlow
• Project one: RNN sentiment analysis of the IMDb movie review dataset
• Project two: RNN character-level language modeling with LSTM cells, using text data from Jules Verne's The Mysterious Island
• Using gradient clipping to avoid exploding gradients
• Introducing the Transformer model and understanding...