Adversarial training

The brilliant idea of adversarial training, proposed by Goodfellow and others (in Generative Adversarial Networks, Goodfellow I. J., Pouget-Abadie J., Mirza M., Xu B., Warde-Farley D., Ozair S., Courville A., Bengio Y., arXiv:1406.2661 [stat.ML]), ushered in a new generation of generative models that immediately outperformed the majority of existing algorithms. All of the derived models are based on the same fundamental concept of adversarial training, which is an approach partially inspired by game theory.

Let's suppose that we have a data generating process, p_data(x), that represents an actual data distribution and a finite number of samples that we suppose are drawn from p_data:

Our goal is to train a model called a generator, whose distribution must be as close as possible to p_data. This is the trickiest part of the algorithm, because instead...