Implementing Custom Layers

Previously, you looked at implementing your own custom loss function with either the TensorFlow functional API or the subclassing approach. These concepts can also be applied to creating custom layers for a deep learning model. In this section, you will build a ResNet module from scratch.

Introduction to ResNet Blocks

Residual neural network, or ResNet, was first proposed by Kaiming He in his paper Deep Residual Learning for Image Recognition in 2015. He introduced a new concept called a residual block that tackles the problem of vanishing gradients, which limits the ability of training very deep networks (with a lot of layers).

A residual block is composed of multiple layers. But instead of having a single path where each layer is stacked and executed sequentially, a residual block contains two different paths. The first path has two different convolution layers. The second path, called the skip connection, takes the input and forwards it to the...