Generative models

Generative models are models that learn to create data similar to data it is trained on. We saw one example of a generative model that learns to write prose similar to Alice in Wonderland in Chapter 6, Recurrent Neural Network — RNN. In that example, we trained a model to predict the 11th character of text given the first 10 characters. Yet another type of generative model is generative adversarial models (GAN) that have recently emerged as a very powerful class of models—you saw examples of GANs in Chapter 4, Generative Adversarial Networks and WaveNet. The intuition for generative models is that it learns a good internal representation of its training data, and is therefore able to generate similar data during the prediction phase.

Another perspective on generative models is the probabilistic one. A typical classification or regression network, also called a discriminative model, learns a function that maps the input data X to some label or output y, that is, these models...