Using Captum to interpret models
Captum [3] is an open source model interpretability library built by Meta on top of PyTorch, and it is currently (at the time of writing) under active development. In this section, we will use the handwritten digits classification model that we had trained in the preceding section. We will also use some of the model interpretability tools offered by Captum to explain the predictions made by this model. The full code for the following exercise can be found in our GitHub repository [4].
Setting up Captum
The model training code is similar to the code shown in the Training the handwritten digits classifier – a recap section of this chapter. In the following steps, we will use the trained model and a sample image to understand what happens inside the model while making a prediction for the given image:
- There are a few extra imports related to Captum that we need to perform in order to use Captum’s built-in model interpretability...
