Evaluating the sparse decomposition

The sparse autoencoder is also known as over-complete representation and has a higher number of nodes in the hidden layer. The sparse autoencoders are usually executed with the sparsity parameter (regularization), which acts as a constraint and restricts the node to being active. The sparsity can also be assumed as nodes dropout caused due to sparsity constraints. The loss function for a sparse autoencoder consists of a reconstruction error, a regularization term to contain the weight decay, and KL divergence for sparsity constraint. The following representation gives a very good illustration of what we are talking about:

Getting ready

1. The dataset is loaded and set up.
2. Install and load the autoencoder package using the following script:

install.packages("autoencoder")
require(autoencoder)

How to do it...

1. The standard autoencoder code of TensorFlow can easily be extended to the sparse autoencoder module by updating the cost function. This section will introduce...