Using the Hugging Face Hub for pre-trained models

In the previous section, we used a pre-trained BERT model as an example to demonstrate the interface between the transformers and torch libraries. In that example, we used the Hugging Face Hub [5] to download and then load a pre-trained BERT model. In this section, we will explore using the Hugging Face Hub to load pre-trained models further and demonstrate how to use the Hugging Face Hub via a Python library and via the Hugging Face website. All the code for this section is available on GitHub [6].

Note

You need to use an API token to access many of the models available on the Hugging Face Hub. You can access your API token from the following page: https://huggingface.co/settings/tokens.

To use the Hugging Face Hub Python library, we need to install it with the following command:

pip install huggingface_hub

Once it is installed, we can run the following command to import the library and fetch all...