Expectation-Maximization (EM)
The EM was originally introduced to estimate the maximum likelihood in the case of incomplete data [4:7]. The EM algorithm is an iterative method to compute the model features that maximize the likely estimate for observed values, considering unobserved values.
The iterative algorithm consists of computing:
- The expectation, E, of the maximum likelihood for the observed data by inferring the latent values (E-step)
- The model features that maximize the expectation E (M-step)
The EM algorithm is applied to solve clustering problems by if each latent variable follows a Normal or Gaussian distribution. This is similar to the K-means algorithm for which the distance of each data point to the center of each cluster follows a Gaussian distribution [4:8]. Therefore, a set of latent variables is a mixture of Gaussian distributions.
Gaussian mixture model
Latent variables, Zi can be visualized as the behavior (or symptoms) of a model (observed), X, for which Z are the root-cause...
