In this chapter, you will learn how to implement a model that is used in the paper Adversarial Generation of Natural Language by Rajeswar et al. This model was first described in the paper Improved Training of Wasserstein GANs by Gulrajani et al. It is capable of generating short discrete sequences with small vocabularies.
We will first address language generation as a problem of conditional probability, in which we want to estimate the probability of the next token given the previous tokens. Then will address the challenges involved in training models for discrete sequences using GANs.
After this introduction to language generation, you will learn how to implement the model described in the paper by Rajeswar et al. and train it on the Google 1 Billion Word Dataset. We will train two separate models: one to generate sequences of characters...
