Some of the successful and popular architectures, such as ResNet and Inception, have shown the importance of deeper and wider networks. ResNet uses shortcut connections to build deeper networks. DenseNet takes it to a new level by introducing connections from each layer to all other subsequent layers, that is a layer where one could receive all the feature maps from the previous layers. Symbolically, it would look like the following:
The following figure describes what a five-layer dense block would look like:
Image source: https://arxiv.org/abs/1608.06993
There is a DenseNet implementation of torchvision (https://github.com/pytorch/vision/blob/master/torchvision/models/densenet.py). Let's look at two major functionalities, _DenseBlock and _DenseLayer.
