The researchers of BERTSUM have used the CNN/DailyMail news dataset. The CNN/DailyMail dataset consists of news articles along with their highlights. We split the CNN/DailyMail news dataset into train and test sets. We train the model using the train set and evaluate it on the test set.
The following shows the ROUGE score of an extractive summarization task using BERTSUM with a classifier, a transformer, and LSTM. We can observe that BERTSUM with the transformer performs slightly better than the others:
The following shows the ROUGE score of the abstractive summarization task using BERTSUMABS:
Thus, we have learned how to fine-tune the BERT model for abstractive and extractive summarization tasks. In the next section, we will see how to train the BERTSUM model.
