Answer to question 1: The following hyperparameters are very important and must be tuned to achieve optimized results:
- Dropout is used to randomly off certain neurons (that is, feature detectors) to prevent overfitting
- Learning rate optimization—Adagrad can be used for feature-specific learning rate optimization
- Regularization—L1 and/or L2 regularization
- Gradient normalization and clipping
- Finally, apply batch normalization to reduce internal covariate shift in training
Now, for dropout, we can add dropout in each convolutional and dense layer and in case of overfitting, the model is specifically adjusted to the training dataset, so it will not be used for generalization. Therefore, although it performs well on the training set, its performance on the test dataset and subsequent tests is poor because it lacks the generalization property.
Anyway...
