Evaluating embeddings – quantitative analysis
A few words might be enough to indicate that the quantitative analysis of embeddings is also possible.
Some word similarity benchmarks propose human-based distances between concepts: Simlex999 (Hill et al., 2016), Verb-143 (Baker et al., 2014), MEN (Bruni et al., 2014), RareWord (Luong et al., 2013), and MTurk- 771 (Halawi et al., 2012).
Our similarity distance between embeddings can be compared to these human distances, using Spearman's rank correlation coefficient to quantitatively evaluate the quality of the learned embeddings.