Autoencoders

Autoencoders are a specifically designed neural network architecture that aims to compress the input information into lower dimensional space in an efficient yet descriptive manner. Autoencoder networks can be decomposed into two individual sub-networks or stages: an encoding stage and a decoding stage.

The following is a simplified autoencoder model using the CIFAR-10 dataset:

Figure 5.27: Simple autoencoder network architecture

The first, or encoding, stage takes the input information and compresses it through a subsequent layer that has fewer units than the size of the input sample. The latter stage, that is, the decoding stage, then expands the compressed form of the image and aims to return the compressed data to its original form. As such, the inputs and desired outputs of the network are the same; the network takes, say, an image in the CIFAR-10 dataset and tries to return the same image. This network architecture is shown in the preceding...