Sparse autoencoder

Autoencoders are fundamentally unsupervised learning models. They are widely used for feature extraction and dimension reduction. The simplest autoencoders are directly derived from the feed-forward neural network (see the Feed-forward neural network section of Chapter 10, Multilayer Perceptron).

The autoencoder attempts to reconstruct its input and therefore, the output and input layer have the same number of nodes or neurons. A conventional neural network such as the multilayer perceptron predicts a target or output vector y from an input vector x. An autoencoder predicts the output layer as the input layer x which constrains the network topology to be symmetric.

Undercomplete autoencoder

One useful application of autoencoders is the extraction of the features that are relevant to the training set (dimension reduction). The hidden layers are stacked using a symmetric pattern along a central hidden layer as described in the following diagram:

Architecture overview of an...