Analyzing sentiment

In an age where more and more is generated, and especially where every individual can post his or her opinion on the internet, the value of automatically analyzing these posts with high accuracy on a large scale is important for businesses and politics. In Chapter 4, Recurrent and Recursive Neural Networks, we've already shown how to apply RNNs with LSTM units to classify short sentences, such as movie reviews. In the following recipe, we will increase the complexity by classifying the sentiments of Twitter messages. We do this by predicting both binary classes and fine-grained classes.

How to do it...

1. We start by all the libraries as follows:

from nltk.tokenize import word_tokenize 
from nltk.stem import WordNetLemmatizer 
import numpy as np 
import random 
import pickle 
from collections import Counter 

import tensorflow as tf

2. Next, we process the English sentences with the nltk package. We start by defining the functions we need for preprocessing:

 

lemmatizer = WordNetLemmatizer...