Fine-tuning the parameters of the autoencoder
The autoencoder involves a couple of parameters to tune, depending on the type of autoencoder we are working on. The major parameters in an autoencoder include the following:
- Number of nodes in any hidden layer
- Number of hidden layers applicable for deep autoencoders
- Activation unit such as sigmoid, tanh, softmax, and ReLU activation functions
- Regularization parameters or weight decay terms on hidden unit weights
- Fraction of the signal to be corrupted in a denoising autoencoder
- Sparsity parameters in sparse autoencoders that control the expected activation of neurons in hidden layers
- Batch size, if using batch gradient descent learning; learning rate and momentum parameter for stochastic gradient descent
- Maximum iterations to be used for the training
- Weight initialization
- Dropout regularization if dropout is used
These hyperparameters can be trained by setting the problem as a grid search problem. However, each hyperparameter combination requires training...
