Introducing AlexNet

The first model we'll discuss is the winner of the 2012 ImageNet Large Scale Visual Recognition Challenge (ILSVRC, or simply ImageNet). It's nicknamed AlexNet (ImageNet Classification with Deep Convolutional Neural Networks, https://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks.pdf), after one of its authors, Alex Krizhevsky. Although this model is rarely used nowadays, it's an important milestone in contemporary deep learning.

The following diagram shows the network architecture:

The AlexNet architecture. The original model was split in two, so it can fit on the memory of two GPUs

The model has five cross-correlated convolutional layers, three overlapping max pooling layers, three fully connected layers, and ReLU activations. The output is a 1,000-way softmax (one for each ImageNet class). The first...