Within the field of Natural language understanding (NLU), Natural language generation is one of the most challenging tasks in machine learning. Broadly speaking, it is easier to estimate the parameters of a discriminative model than a generative model. On Quora (https://www.quora.com/Why-are-generative-models-harder-to-create-than-discriminative-models), Ian Goodfellow gives a good informal explanation that can be generalized to language:
Can you look at a painting and recognize it as being the Mona Lisa? You probably can. That's discriminative modeling. Can you paint the Mona Lisa yourself? You probably can't. That's generative modeling.
The task of modeling language has been approached with rule-based and data-based models, including deep learning. As Ziang Xie has informally explained in his practical guide for neural text...
