Other Regularization Methods

In this section, you will learn briefly about some other regularization techniques that are commonly used and have been shown to be effective in deep learning. It is important to keep in mind that regularization is a wide-ranging and active research field in machine learning. As a result, covering all available regularization methods in one chapter is not possible (and most likely not necessary, especially in a book on applied deep learning). Therefore, in this section, we will briefly cover three more regularization methods, called early stopping, data augmentation, and adding noise. You will learn briefly about their underlying ideas, and you'll gain a few tips and recommendations on how to use them.

Early Stopping

We discussed earlier in this chapter that the main assumption in machine learning is that there is a true function/process that produces training examples. However, this process is unknown and there is no explicit way to find it. Not only is there...