Variational autoencoders

To understand VAEs, let's talk about regular autoencoders first. An autoencoder is a feed-forward neural network that tries to reproduce its input. In other words, the target value (label) of an autoencoder is equal to the input data, yⁱ = xⁱ, where i is the sample index.We can formally say that it tries to learn an identity function, (a function that repeats its input). Since our "labels" are just the input data, the autoencoder is an unsupervised algorithm. The following diagram represents an autoencoder:

An autoencoder

An autoencoder consists of an input, hidden (or bottleneck), and output layers. Although it's a single network, we can think of it as a virtual composition of two components:

• Encoder: Maps the input data to the network's internal representation. For the sake of simplicity, in this example the encoder is a...