Generative models

We will look into the various approaches of generative models in the following sections.

Autoregressive models

Autoregressive models estimate the conditional distribution of some data , given some other values of y. For example, in image synthesis, it estimates the conditional distribution of pixels given surrounding or previous pixels; in audio synthesis, it estimates the conditional distribution of audio samples given previous audio samples and spectrograms.

In its simplest linear form, with dependency on the previous time-step only and time-invariant bias term, an autoregressive model can be defined with the following equation:

is a constant term that represents the model's bias, represents the...