Text generation
The first GPT model was introduced in a 2018 paper by Radford et al. from OpenAI – it demonstrated how a generative language model can acquire knowledge and process long-range dependencies thanks to pretraining on a large, diverse corpus of contiguous text. Two successor models (trained on more extensive corpora) were released in the following years: GPT-2 in 2019 (1.5 billion parameters) and GPT-3 in 2020 (175 billion parameters). In order to strike a balance between demonstration capabilities and computation requirements, we will be working with GPT-2 – as of the time of writing, access to the GPT-3 API is limited.
We'll begin by demonstrating how to generate your own text based on a prompt given to the GPT-2 model without any finetuning.
How do we go about it?
We will be making use of the excellent Transformers library created by Hugging Face (https://huggingface.co/). It abstracts away several components of the building process, allowing...
