Implementing ViTs

We will implement ViTs on the cats vs dogs dataset that we leveraged in Chapters 4 and 5:

The following code is available in the ViT_Image_classification.ipynb file in the Chapter15 folder of this book’s GitHub repository at https://bit.ly/mcvp-2e.

1. Install and import the required packages:

   %pip install -U torch-snippets transformers kaggle
   from torch_snippets import *
   from transformers import ViTModel, ViTConfig
   from torch.optim import Adam
   model_checkpoint = 'google/vit-base-patch16-224-in21k'
   

Note that we will be leveraging the pre-trained ViT model (checkpoint location provided above).

2. Import the dataset:

   %%writefile kaggle.json
   {"username":"xx", "key":"xx"}
   !mkdir -p ~/.kaggle
   !cp kaggle.json ~/.kaggle/
   !chmod 600 /root/.kaggle/kaggle.json
   !kaggle datasets download -d tongpython/cat-and-dog
   !unzip cat-and-dog.zip
   

3. Specify...