Removing stopwords

Stopwords are counted as noise in text analysis. Any text paragraph has to have verbs, articles, and propositions. These are all considered stop words. Stop words are necessary for human conversation but they don't make many contributions in text analysis. Removing stopwords from text is called noise elimination.

Let's see how to remove stopwords using NLTK:

# import the nltk stopwords
from nltk.corpus import stopwords

# Load english stopwords list
stopwords_set=set(stopwords.words("english"))

# Removing stopwords from text
filtered_word_list=[]
for word in tokenized_words:
    # filter stopwords
    if word not in stopwords_set:
        filtered_word_list.append(word)

# print tokenized words
print("Tokenized Word List:", tokenized_words)

# print filtered words
print("Filtered Word List:", filtered_word_list)

This results in the following output:

Tokenized Word List: ['Taj', 'Mahal', 'is', 'one', &apos...