Performing text analysis

The field of Natural Language Processing (NLP) is used for many different tasks including text searching, language translation, sentiment analysis, speech recognition, and classification to mention a few. Processing text is difficult due to a number of reasons, including the inherent ambiguity of natural languages.

 

There are several different types of processing that can be performed such as:

• Identifying Stop words: These are words that are common and may not be necessary for processing
• Name Entity Recognition (NER): This is the process of identifying elements of text such as people's names, location, or things
• Parts of Speech (POS): This identifies the grammatical parts of a sentence such as noun, verb, adjective, and so on
• Relationships: Here we are concerned with identifying how parts of text are related to each other, such as the subject and object of a sentence

As with most data science problems, it is important to preprocess and clean text. In Chapter 9, Text Analysis...