Latent Dirichlet Allocation

The most common technique currently in use for topic modeling of text, and the one that the Facebook researchers used in their 2013 paper, is called Latent Dirichlet Allocation (LDA).

Tip

Many people wonder how to pronounce Dirichlet in English. The most common pronunciation I have heard is DEER-uh-shlay, and I have also heard DEER-uh-klay a few times.

LDA was first proposed for text topic extraction by David Blei, Andrew Ng, and Michael Jordan in a 2003 paper entitled simply Latent Direchlet Allocation, available from the Journal of Machine Learning Research at http://www.jmlr.org/papers/volume3/blei03a/blei03a.pdf. Blei also wrote a good follow-up article in 2012 for the Communications of the ACM about LDA and some new variants and improvements for it. This later article is written in very accessible language and is available for download at https://www.cs.princeton.edu/~blei/papers/Blei2012.pdf.

The first thing we should know about LDA is that it is a probabilistic...