Performing preprocessing of textual data and extraction of sentiments

In this section, we will use Jane Austen's bestselling novel Pride and Prejudice, published in 1813, for our textual data preprocessing analysis. In R, we will use the tidytext package by Hadley Wickham to perform tokenization, stop word removal, sentiment extraction using predefined sentiment lexicons, term frequency - inverse document frequency (tf-idf) matrix creation, and to understand pairwise correlations among n-grams.

In this section, instead of storing text as a string or a corpus or a document term matrix (DTM), we process them into a tabular format of one token per row.

How to do it...

Here is how we go about preprocessing:

1. Load the required packages:

load_packages=c("janeaustenr","tidytext","dplyr","stringr","ggplot2","wordcloud","reshape2","igraph","ggraph","widyr","tidyr") 
lapply(load_packages, require, character.only = TRUE) 

2. Load the Pride and Prejudice dataset. The line_num attribute is analogous to the line...