Discussing other algorithm-based techniques
In this section, we’ll explore a diverse set of algorithm-level techniques that we haven’t covered so far. Intriguingly, these methods – from regularization techniques that mitigate overfitting to Siamese networks skilled in one-shot and few-shot learning, to deeper neural architectures and threshold adjustments – also have a beneficial side effect: they can occasionally mitigate the impact of class imbalance.
Regularization techniques
The paper from S. Alshammari et al. [14] found that well-known regularization techniques such as L2-regularization and the MaxNorm constraint are quite helpful in long-tailed recognition. The paper proposes to do these only at the last layer of classification (sigmoid or softmax, for example). Here are their findings:
- L2-regularization (also called weight decay) generally keeps the weights in check and helps the model generalize better by preventing the model from overfitting...
