In this section, we are going to analyze an algorithm that can be efficiently employed for online linear classifications. In fact, one of the problems with other methods is that when new samples are collected, the whole model must be retrained. The main idea proposed by Crammer et al. (in Online Passive-Aggressive Algorithms, Crammer K., Dekel O., Keshet J., Shalev-Shwartz S., Singer Y., Journal of Machine Learning Research 7 (2006) 551–585) is to train a model incrementally, allowing modifications of the parameters only when needed, while discarding all the updates that don't alter the equilibrium. In the original paper, three variants were proposed. In this description, we are considering the one called PA-II (which is the most flexible).
For simplicity, in this description we are assuming bipolar outputs (-1, +1); however, there are...
