Using GPT models as text generators

Using libraries such as Hugging Face’s transformers or openai together with PyTorch, we can load most of the latest advanced transformer models for performing various tasks such as language modeling, text classification, machine translation, and so on. We demonstrated how to do so in Chapter 5, Advanced Hybrid Models.

In this section, we will first load the GPT-2-language model using transformers. We will then extend this 1.5-billion-parameters model so that we can use it as a text generator. Then, we will explore the various strategies we can follow to generate text from a pre-trained language model and use PyTorch to demonstrate those strategies.

Finally, we will load the 175-billion-parameters GPT-3 model using openai and demonstrate its capability to generate realistic natural language.

Out-of-the-box text generation with GPT-2

In the form of an exercise, we will load the GPT-2 language model using the transformers library...