With research into neural networks once again back on track, deep learning started growing, until a major breakthrough in 2012, which finally gave it its contemporary prominence. Since the publication of ImageNet, a competition (ImageNet Large Scale Visual Recognition Challenge (ILSVRC)—image-net.org/challenges/LSVRC) has been organized every year for researchers to submit their latest classification algorithms and compare their performance on ImageNet with others. The winning solutions in 2010 and 2011 had classification errors of 28% and 26% respectively, and applied traditional concepts such as SIFT features and SVMs. Then came the 2012 edition, and a new team of researchers reduced the recognition error to a staggering 16%, leaving all the other contestants far behind.
In their paper describing this achievement (Imagenet Classification with Deep Convolutional Neural Networks, NIPS, 2012), Alex Krizhevsky, Ilya Sutskever, and Geoff Hinton presented what...
