Text summarization with T5

NLP summarizing tasks extract succinct parts of a text. This section will start by presenting the Hugging Face resources we will use in this chapter. Then, we will initialize a T5-large transformer model. Finally, we will see how to use T5 to summarize any document, including legal and corporate documents.

Let’s begin by introducing Hugging Face’s framework.

Hugging Face

Hugging Face designed a framework to implement transformers at a higher level. For example, we already used Hugging Face to fine-tune a BERT model in Chapter 5, Diving into Fine-Tuning Through BERT, and train a RoBERTa model in Chapter 6, Pretraining a Transformer from Scratch through RoBERTa.

This chapter will use Hugging Face’s framework again to implement a T5-large model.

Selecting a Hugging Face transformer model

In this subsection, we will choose the T5 model we will implement in this chapter.

A wide range of models can be found on the...