Hard example mining
Hard example mining is a technique in deep learning that forces the model to pay more attention to these difficult examples, and to prevent overfitting to the majority of the samples that are easy to predict. To do this, hard example mining identifies and selects the most challenging samples in the dataset and then backpropagates the loss incurred only by those challenging samples. Hard example mining is often used in computer vision tasks such as object detection. Hard examples can be of two kinds:
- Hard positive examples are the correctly labeled examples with low prediction scores
- Hard negative examples are incorrectly labeled examples with high prediction scores, which are obvious mistakes made by the model
The term “mining” refers to the process of finding such examples that are “hard.” The idea of hard negative mining is not really new and is quite similar to the idea of boosting, on which the popular algorithms...
