Custom layers
In the previous section, I briefly mentioned the nn.Module class as a base parent for all NN building blocks exposed by PyTorch. It’s not just a unifying parent for the existing layers — it’s much more than that. By subclassing the nn.Module class, you can create your own building blocks, which can be stacked together, reused later, and integrated into the PyTorch framework flawlessly.
At its core, the nn.Module provides quite rich functionality to its children:
-
It tracks all submodules that the current module includes. For example, your building block can have two feed-forward layers used somehow to perform the block’s transformation. To keep track of (register) the submodule, you just need to assign it to the class’s field.
-
It provides functions to deal with all parameters of the registered submodules. You can obtain a full list of the module’s parameters (parameters...
