One way to build mixture models is to consider a finite weighted mixture of two or more distributions. This is known as a finite mixture model. Thus, the probability density of the observed data is a weighted sum of the probability density for subgroups of the data:
![](https://static.packt-cdn.com/products/9781789341652/graphics/assets/7491d3bb-b383-49e6-8960-e9579059a773.png)
Here, is the weight of each component (or class). We can interpret
as the probability of the component
, thus its values are restricted to the interval [0, 1] and
. The components
can be virtually anything we may consider useful from simple distributions, such as a Gaussian or a Poisson, to more complex objects, such as hierarchical models or neural networks. For a finite mixture model,
is a finite number (usually, but not necessary, a small number
). In order to fit a finite mixture model, we need to provide a value of
, either because we really know the correct value beforehand, or because...