Autoencoder architectures

As we mentioned, a typical autoencoder consists of three parts. Let's explore these three parts in more detail. To motivate you, we are not going to reinvent the wheel here in this chapter. The encoder-decoder part is nothing but a fully connected neural network, and the code part is another neural network but it's not fully connected. The dimensionality of this code part is controllable and we can treat it as a hyperparameter:

Figure 3: General encoder-decoder architecture of autoencoders

Before diving into using autoencoders for compressing the MNIST dataset, we are going to list the set of hyperparameters that we can use to fine-tune the autoencoder model. There are mainly four hyperparameters:

1. Code part size: This is the number of units in the middle layer. The lower the number of units we have in this layer, the more compressed the representation...