3. Ground truth anchor boxes
From Figure 11.2.3, it appears that given an object bounding box, there are many ground truth anchor boxes that can be assigned to an object. In fact, just for the illustration in Figure 11.2.3, there are already 3 anchor boxes. If all anchor boxes per region are considered, there are 6 x 6 = 36 ground truth boxes just for . Using all 9,648 anchor boxes is obviously excessive. Only one of all anchor boxes should be associated with the ground truth bounding box. All other anchor boxes are background anchor boxes. What is the criterion for choosing which one should be considered the ground truth anchor box for an object in the image?
The basis for choosing the anchor box is called Intersection over Union (IoU). IoU is also known as Jaccard index. IoU is illustrated in Figure 11.3.1. Given 2 regions, an object bounding box, B0 and an anchor box, A1, IoU is equal to the area of overlap divided by the area of the combined regions: