Multilabel text classification

We had already solved multi-class text classification in this chapter, where a single label is assigned to each text. Now, we will discuss multi-label classification, where a text can have multiple labels. This is common in NLP applications such as news classification. For instance, news can be related to sports and health at the same time. The following figure depicts multi-label classification:

Figure 5.13 – Multi-label classification scheme

Now, we will dive into how to develop a pipeline to apply multi-label classification. To do so, we will take the PubMed dataset, which comprises around 50,000 research articles. The dataset has multiple labels for the articles. Biomedical experts manually annotated these articles with MeSH labels, and each article has been described based on the combination of 14 MeSH labels.

Now, we will start by importing the libraries:

import torch, numpy as np, pandas as pd
from datasets...