Introduction

Neural networks aim to find a non-linear relationship between input X with output y, as y=f(x). An autoencoder is a form of unsupervised neural network which tries to find a relationship between features in space such that h=f(x), which helps us learn the relationship between input space and can be used for data compression, dimensionality reduction, and feature learning.

Note

An autoencoder consists of an encoder and decoder. The encoder helps encode the input x in a latent representation y, whereas a decoder converts back the y to x. Both the encoder and decoder possess a similar representation of form.

Here is a representation of a one layer autoencoder:

The coder encodes input X to h under a hidden layer contain, whereas the decoder helps to attain the original data from encoded output h. The matrices W_e and W_d represent the weights of the encoder and decoder layers, respectively. The function f is the activation function.

An illustration of an autoencoder is shown in the following...