Building computer vision classifiers using conformal prediction

Let’s illustrate the application of conformal prediction to computer vision in practice. We will use a notebook from the book repository available at https://github.com/PacktPublishing/Practical-Guide-to-Applied-Conformal-Prediction/blob/main/Chapter_09.ipynb. This notebook extensively uses notebooks from Anastasios Angelopolous’ Conformal Prediction repo at https://github.com/aangelopoulos/conformal-prediction.

After loading the data, set up the problem and define the desired coverage and the number of points in the calibration set:

n_cal = 1000
alpha = 0.1

The softmax scores were split into the calibration and test datasets, obtaining calibration and test labels:

idx = np.array([1] * n_cal + [0] * (smx.shape[0]-n_cal)) > 0
np.random.seed(42)
np.random.shuffle(idx)
cal_smx, test_smx = smx[idx,:], smx[~idx,:]
cal_labels, test_labels = labels[idx], labels[~idx]

The test dataset contains 49...