What is a word representation or meaning?

What is meant by the word meaning? This is more of a philosophical question than a technical one. So, we will not try to discern the most proper answer for this question, but accept a more modest answer, that is, meaning is the idea or the representation conveyed by a word. Since the primary objective of NLP is to achieve human-like performance in linguistic tasks, it is sensible to explore principled ways of representing words for machines. To achieve this, we will use algorithms that can analyze a given text corpus and come up with good numerical representations of words (that is, word embeddings), such that words that fall within similar contexts (for example, one and two, I and we) will have similar numerical representations compared with words that are irrelevant (for example, cat and volcano).

First, we will discuss some classical approaches to achieve this and then move on to understanding more sophisticated recent methods that use neural...