One way to build mixture models is to consider a finite weighted mixture of two or more distributions. This is known as a finite mixture model. Thus, the probability density of the observed data is a weighted sum of the probability density for subgroups of the data:
Here, is the weight of each component (or class). We can interpret as the probability of the component , thus its values are restricted to the interval [0, 1] and . The components can be virtually anything we may consider useful from simple distributions, such as a Gaussian or a Poisson, to more complex objects, such as hierarchical models or neural networks. For a finite mixture model, is a finite number (usually, but not necessary, a small number ). In order to fit a finite mixture model, we need to provide a value of , either because we really know the correct value beforehand, or because...