Variational autoencoders

A variational autoencoder (VAE) is a generative model proposed by Kingma and Wellin (in their work Auto-Encoding Variational Bayes, arXiv:1312.6114 [stat.ML]) that partially resembles a standard autoencoder, but it has some fundamental internal differences. The goal, in fact, is not finding an encoded representation of a dataset, but determining the parameters of a generative process that is able to yield all possible outputs given an input data-generating process.

Let's take the example of a model based on a learnable parameter vector θ and a set of latent variables z that have a probability density function p(z;θ). Our goal can therefore be expressed as the research of the θparametersthat maximize the likelihood of the marginalized distribution p(x;θ) (obtained through the integration of the joint probability p(x,z;θ)):

If this problem could be easily solved in closed form, a large set of samples drawn from the p(x)data generating processwould be enough to find a...