What is deep learning?
In 2012, Alex Krizhevsky, Ilya Sutskever, and Geoff Hinton published an article titled ImageNet Classification with Deep Convolutional Neural Networks in Proceedings of Neural Information Processing Systems (NIPS) (2012) and, at the end of their paper, they wrote:
"It is notable that our network's performance degrades if a single convolutional layer is removed. For example, removing any of the middle layers results in a loss of about 2% for the top-1 performance of the network. So the depth really is important for achieving our results."
In this milestone paper, they clearly mention the importance of the number of hidden layers present in deep networks. Krizheysky, Sutskever, and Hilton talk about convolutional layers, and we will not discuss them until Chapter 5, Image Recognition, but the basic question remains: What do those hidden layers do?
A typical English saying is a picture is worth a thousand words. Let's use this approach to understand...
