Dr Joshua Eckroth on performing Sentiment Analysis on social media platforms using CoreNLP

Sentiment analysis using Natural Language Processing

A more modern approach may be found in Stanford's open source CoreNLP project . CoreNLP supports a wide range of NLP processing such as sentence detection, word detection, part-of-speech tagging, named-entity recognition (finding names of people, places, dates, and so on), and sentiment analysis. Several NLP features, such as sentiment analysis, depend on prior processing including sentence detection, word detection, and part-of-speech tagging. As described in the following text, a sentence's dependency tree, which shows the subject, object, verbs, adjectives, and prepositions of a sentence, is critical for sentiment analysis. CoreNLP's sentiment analysis technique has been shown to achieve 85.4% accuracy for detecting positive/negative sentiment of sentences. Their technique is state-of-the-art and has been specifically designed to better handle negation in various places in a sentence, a limitation of simpler sentiment analysis techniques as previously described.

CoreNLP's sentiment analysis uses a technique known as recursive neural tensor networks (RNTN) (Here, a sentence or phrase is parsed into a binary tree, as seen in Figure 1. Every node is labeled with its part-of-speech: NP (noun phrase), VP (verb phrase), NN (noun), JJ (adjective), and so on. Each leaf node, that is, each word node, has a corresponding word vector. A word vector is an array of about 30 numbers (the actual size depends on a parameter that is determined experimentally). The values of the word vector for each word are learned during training, as is the sentiment of each individual word. Just having word vectors will not be enough since we have already seen how sentiment cannot be accurately determined by looking at words independently of their context.

The next step in the RNTN procedure collapses the tree, one node at a time, by calculating a vector for each node based on its children. The bottom-right node of the figure, the NP node with children own and crashes, will have a vector that is the same size of the word vectors but is computed based on those child word vectors. The computation multiplies each child word vector and sums the results. The exact multipliers to use are learned during training. The RNTN approach, unlike prior but similar tree collapsing techniques, uses a single combiner function for all nodes. Ultimately, the combiner function and the word vectors are learned simultaneously using thousands of example sentences with the known sentiment.

Figure 1: CoreNLP's dependency tree parse of the sentence, "Self-driving car companies should not be allowed to investigate their own crashes"

The dependency tree from the preceding figure has twelve leaf nodes and twelve combiner nodes. Each leaf node has an associated word vector learned during training. The sentiment of each leaf node is also learned during training. Thus, the word crashes, for example, has a neutral sentiment with 0.631 confidence, while the word not has negative sentiment with 0.974 confidence. The parent node of allowed and the phrase to investigate their own crashes has a negative sentiment, confidence 0.614, even though no word or combiner node among its descendants have anything but neutral sentiment. This demonstrates that the RNTN learned a complex combiner function that operates on the word vectors of its children and not just a simple rule such as, If both children are neutral, then this node is neutral, or if one child is neutral, but one is positive, this node is positive, ....

The sentiment values and confidence of each node in the tree is shown in the output of CoreNLP shown in the following code block. Note that sentiment values are coded:

• 0 = very negative
• 1 = negative
• 2 = neutral
• 3 = positive
• 4 = very positive

(ROOT|sentiment=1|prob=0.606

 (NP|sentiment=2|prob=0.484

   (JJ|sentiment=2|prob=0.631 Self-driving)

(NP|sentiment=2|prob=0.511

  (NN|sentiment=2|prob=0.994 car)

  (NNS|sentiment=2|prob=0.631 companies)))

 (S|sentiment=1|prob=0.577

(VP|sentiment=2|prob=0.457

     (VP|sentiment=2|prob=0.587

    (MD|sentiment=2|prob=0.998 should)

    (RB|sentiment=1|prob=0.974 not))

     (VP|sentiment=1|prob=0.703

       (VB|sentiment=2|prob=0.994 be)

       (VP|sentiment=1|prob=0.614

     (VBN|sentiment=2|prob=0.969 allowed)

         (S|sentiment=2|prob=0.724

           (TO|sentiment=2|prob=0.990 to)

           (VP|sentiment=2|prob=0.557

             (VB|sentiment=2|prob=0.887 investigate)

             (NP|sentiment=2|prob=0.823

               (PRP|sentiment=2|prob=0.997 their)

               (NP|sentiment=2|prob=0.873

                 (JJ|sentiment=2|prob=0.996 own)

                 (NNS|sentiment=2|prob=0.631 crashes))))))))

   (.|sentiment=2|prob=0.997 .)))

We see from these sentiment values that allowed to investigate their own crashes is labeled with negative sentiment. We can investigate how CoreNLP handles words such as allowed and not by running through a few variations. These are shown in the following table:

Table 1: Variations of a sentence with CoreNLP's sentiment analysis

It is clear from Table 1 that the phrase investigates their own crashes is not contributing strongly to the sentiment of the whole sentence. The verb, allowed, happy, or willing can dramatically change the sentiment. The modifier not can flip the sentiment, though curiously not unwilling is still considered negative.

We should be particularly careful to study CoreNLP's sentiment analysis with sentence fragments and other kinds of invalid English that is commonly seen on Twitter. For example, the Twitter API will deliver phrases such as, Ford's self-driving car network will launch 'at scale' in 2021 - Ford hasn't been shy about... with the ... in the actual tweet. CoreNLP labels this sentence as negative with confidence 0.597.

CoreNLP was trained on movie reviews, so news articles, tweets, and Reddit comments may not match the same kind of words and grammar found in movie reviews. We might have a domain mismatch between the training domain and the actual domain. CoreNLP can be trained on a different dataset but doing so requires that thousands (or 10's or 100's of thousands) of examples with known sentiment are available. Every node in the dependency tree of every sentence must be labeled with a known sentiment. This is very time-consuming. The authors of CoreNLP used Amazon Mechanical Turk to recruit humans to perform this labeling task.

To summarize, in this article, we have demonstrated how to identify the sentiment, or general mood, of the feedback from customers and the general public on social media platforms (e.g. Twitter). We also performed sentiment analysis using a method based on machine learning using CoreNLP.

Master essential AI blueprints to program real-world business applications from the book AI Blueprints by Dr Joshua Eckroth.

How to perform sentiment analysis using Python [Tutorial]

Understanding Sentiment Analysis and other key NLP concepts

Sentiment Analysis of the 2017 US elections on Twitter