Understanding text analysis applications
The inherent nature of language and writing to problems of high dimensionality while analyzing documents. Hence, some of the most widely used textual methods rely on the critical assumption of independence, where the order and direct context of a word are not important. Methods, where word sequence is ignored, are typically labeled as "bag-of-words" techniques.
Textual analysis is a lot more imprecise compared to quantitative analysis. Textual data requires an additional step of translating the text into quantitative measures, which are then used as inputs for various text-based analytics or ML methods. Many of these methods are based on deconstructing a document into a term-document matrix consisting of rows of words and columns of word counts.
In applications using a bag of words, the approach to normalizing the word counts is important as the raw counts directly dependent on the document length. A simple use of proportions can this problem, however...
