Further reading

• On the importance of initialization and momentum in deep learning (http://proceedings.mlr.press/v28/sutskever13.pdf), by Ilya Sutskever et al. This often-referenced conference paper, published in 2013, presents and compares the momentum and NAG algorithms.
• Dropout: A Simple Way to Prevent Neural Networks from Overfitting (http://www.jmlr.org/papers/volume15/srivastava14a/srivastava14a.pdf), by Nitish Srivastava et al. This other conference paper, published in 2014, introduced dropout. It is a great read for those who want to know more about this method and see it applied to several famous computer vision datasets.