Image classification and drawbacks of DNNs

Before we start developing the end-to-end project for image classification using CNN, we need some background studies, such as the drawbacks of regular DNNs, suitability of CNNs over DNNs for image classification, CNN constructions, CNN's different operations, and so on. Although regular DNNs work fine for small images (for example, MNIST, CIFAR-10), it breaks down for larger images because of the huge number of parameters it requires. For example, a 100 x 100 image has 10,000 pixels, and if the first layer has just 1,000 neurons (which already severely restricts the amount of information transmitted to the next layer), this means a total of 10 million connections. And that's just for the first layer.

CNNs solve this problem using partially connected layers. Because consecutive layers are only partially connected and because it heavily reuses its weights, a CNN has far fewer parameters than a fully connected DNN, which makes it much faster to train...