Chapter 5: Recurrent Neural Networks and Sentiment Analysis

In this chapter, we will look at Recurrent Neural Networks (RNNs), a variation of the basic feed forward neural networks in PyTorch that we learned how to build in Chapter 1, Fundamentals of Machine Learning. Generally, RNNs can be used for any task where data can be represented as a sequence. This includes things such as stock price prediction, using a time series of historic data represented as a sequence. We commonly use RNNs in NLP as text can be thought of as a sequence of individual words and can be modeled as such. While a conventional neural network takes a single vector as input to the model, an RNN can take a whole sequence of vectors. If we represent each word in a document as a vector embedding, we can represent a whole document as a sequence of vectors (or an order 3 tensor). We can then use RNNs (and a more sophisticated form of RNN known as Long Short-Term Memory (LSTM) to learn from our data.

In this chapter...