Implementing vanilla autoencoders

An autoencoder comprises of the following two networks:

• Encoder: An encoder encodes its input, , into a hidden representation, . The output of an encoder unit is as follows:

h = g(Wx_i+b)

where, x_i ∈ Rⁿ, W ∈ R^{d x n}, b ∈ R^d.

• Decoder: A decoder reconstructs the input from the hidden representation, h. The output of a decoder unit is as follows:
  

where, W* ∈ R^{n x d}, h ∈ R^d, c ∈ Rⁿ.

An autoencoder neural network tries to reconstruct the original input, , from the encoded representation, , of  dimension to produce an output, , such that approximates to . The network is trained to minimize the reconstruction error (loss function). It is a measure of the difference between the original input and the predicted output,...