Mixture models

Sometimes a process or phenomenon under study cannot be properly described using a single distribution like a Gaussian or a binomial, or any other canonical/pure distribution, but it can be described as a mixture of such distributions. Models that assume the data comes from a mixture of distributions are know as mixture models.

One kind of situation where mixture models arise naturally is when we have a dataset that is better described as a combination of real subpopulations. For example, it makes perfect sense to describe the distribution of heights, in an adult human population, as a mixture of female and male subpopulations. Even more, if we have to deal also with non-adults, we may find it useful to include a third group describing children, probably without needing to make a gender distinction inside this group. Another classical example of a mixture model approach is used to describe a group of handwritten digits. In this case, it also makes perfect sense to use 10 subpopulation...