Training a classification model with native PyTorch

The Trainer class is very powerful, and we have the Hugging Face team to thank for providing such a useful tool. However, in this section, we will fine-tune the pre-trained model from scratch to see what happens under the hood. Let’s get started.

First, let’s load the model for fine-tuning. We will select DistilBert here since it is a small, fast, and cheap version of BERT:

from transformers import DistilBertForSequenceClassification
model = DistilBertForSequenceClassification\
    .from_pre-trained('distilbert-base-uncased')

To fine-tune any model, we need to put it into training mode, as follows:

model.train()

Now, we must load the tokenizer:

from transformers import DistilBertTokenizerFast
tokenizer = DistilBertTokenizerFast.from_pre-trained(
    'bert-base-uncased')

Since the Trainer class organized the entire process for us, we did not deal with optimization...