Variational E-M
Variational E-M is an extension of E-M that incorporates variational inference. In variational E-M, during the “expectation” step, instead of computing the exact posterior distribution of the latent variables as in standard E-M, it approximates this posterior using a simpler distribution from a predefined family. Then, during the “maximization” step, it optimizes the model parameters to maximize a lower bound on the likelihood of the observed data, which is derived from the approximate posterior. Variational E-M iterates between these two steps until convergence, providing a computationally efficient way to perform parameter estimation in complex probabilistic models, especially in Bayesian settings.
Now, let’s describe the variational E-M algorithm in our context:
- The E-step: We get the optimal values of the variational parameters, (γ, ϕ), in Eq. (11) and Eq. (12) for every document in the corpus by assuming...
