As we saw in Chapter 5, Word Embeddings and Distance Measurements for Text, Word2Vec helped in fetching semantic embeddings for word-level representations. However, most of the NLP tasks we deal with are a combination of words or are essentially what we call a paragraph:
How do we fetch paragraph-level embeddings?
One simple mechanism would be to take the word embeddings for the words occurring in the paragraph and average them out to have representations of paragraphs:
Can we do better than averaging word embeddings?
Le and Mikolov extended the idea of Word2Vec to develop paragraph-level embeddings so that paragraphs of differing lengths can be represented by fixed-length vectors. In doing so, they presented the paper Distributed Representations of Sentences and Documents (https://arxiv.org/abs/1405.4053), which aimed at building paragraph-level embeddings. Similar to Word2Vec, the idea here is to predict certain words as well. However, in addition to using word...
