Text completion with GPT-2
This section will clone the OpenAI GPT-2 repository, download the 345M parameter GPT-2 transformer model, and interact with it. We will enter context sentences and analyze the text generated by the transformer. The goal is to see how it creates new content.
This section is divided into 9 steps. Open OpenAI_GPT_2.ipynb in Google Colaboratory. The notebook is in the chapter of the GitHub repository of this book. You will notice that the notebook is also divided into the same 9 steps and cells as this section.
Run the notebook cell by cell. The process is tedious, but the result produced by the cloned OpenAI GPT-2 repository is gratifying.
It is important to note that we are running a low-level GPT-2 model and not a one-line call to obtain a result. We are also avoiding pre-packaged versions. We are getting our hands dirty to understand the architecture of a GPT-2 from scratch. You might get some deprecation messages. However, the effort...
