Exploring n-grams
In our CBOW model, we successfully showed that the meaning of the words is related to the context of the words around it. It is not only our context words that influence the meaning of words in a sentence, but the order of those words as well. Consider the following sentences:
The cat sat on the dog
The dog sat on the cat
If you were to transform these two sentences into a bag-of-words representation, we would see that they are identical. However, by reading the sentences, we know they have completely different meanings (in fact, they are the complete opposite!). This clearly demonstrates that the meaning of a sentence is not just the words it contains, but the order in which they occur. One simple way of attempting to capture the order of words within a sentence is by using n-grams.
If we perform a count on our sentences, but instead of counting individual words, we now count the distinct two-word pairings that occur within the sentences, this is known...
