Understanding Discrete Distributions

Latent Dirichlet Allocation (LDA) is named based on its mathematical concepts. Its name contains the word latent because it finds a similarity between documents in the latent space. It contains the word allocation because it allocates a document to topics. But most of all, it contains the word Dirichlet because it is based on the Dirichlet (pronounced as “Deer-e-kh-let”) distribution.

The Dirichlet distribution belongs to the discrete distribution family, which includes the Bernoulli, binomial, multinomial, beta, and Dirichlet distributions. The binomial and multinomial distributions are already frequently used in many machine-learning models that many of you will be familiar with. The beta distribution is a generalization of the binomial distribution, and the Dirichlet distribution is a generalization of the beta distribution. Further, the beta distribution is the conjugate prior probability distribution for the Bernoulli and binomial...