Shielding against any evasion attack via adversarial training of a robust classifier

In Chapter 8, Visualizing Convolutional Neural Networks, we faced a fruit image classifier that would likely perform poorly in the intended environment of a convenience store self-serve checkout. The abysmal performance on out-of-sample data was due to the classifier being trained on many images of one or two fruits per class, taken from entirely different angles with consistent illumination. It turns out that the variety of angles wasn't as important as the variety of fruit and illumination! The chapter's conclusion called for the training of a network with images representing their intended environment, to make for a more robust model.

For model robustness, training data variety is critical, but only if it represents the intended environment. In statistical terms, it's a question of using samples for training that accurately depict the population so that the model learns to classify...