Chapter 7 - Learning Text Representations

1. In the continuous bag-of-words (CBOW) model, we try to predict the target word given the context word, and in the skip-gram model, we try to predict the context word given the target word.

2. The loss function of the CBOW model is given as follows:

   
3. When we have millions of words in the vocabulary, we need to perform numerous weight updates until we predict the correct target word. It is time-consuming and also not an efficient method. So, instead of doing this, we mark the correct target word as a positive class and sample a few words from the vocabulary and mark it as a negative class, and this is called negative sampling

4. PV-DM is similar to a continuous bag of words model, where we try to predict the target word given a context word. In PV-DM, along with word vectors, we introduce one more vector, called the paragraph vector. As the...