A generative model aims to generate all the values of a phenomenon, both those that can be observed (input) and those that can be calculated from the ones observed (target). We try to understand how such a model can succeed in this goal by proposing a first distinction between generative and discriminative models.
Often, in machine learning, we need to predict the value of a target vector y given the value of an input x vector. From a probabilistic perspective, the goal is to find the conditional probability distribution p(y|x).
The conditional probability of an event y with respect to an event x is the probability that y occurs, knowing that x is verified. This probability, indicated by p(y|x), expresses a correction of expectations for y, dictated by the observation of x.
The most common approach to this problem is to represent the conditional distribution...
