Other Regularization Methods

In this section, you will briefly learn about some other regularization techniques that are commonly used and have been shown to be effective in deep learning. It is important to keep in mind that regularization is a wide-ranging and active research field in machine learning. As a result, covering all the available regularization methods in one chapter is not possible (and most likely not necessary, especially in a book on applied deep learning). Therefore, in this section, we will briefly cover three more regularization methods, called early stopping, data augmentation, and adding noise. You will learn about their underlying ideas and gain a few tips and recommendations on how to use them.

Early Stopping

Earlier in this chapter, we discussed that the main assumption in machine learning is that there is a true function or process that produces training examples. However, this process is unknown and there is no explicit way to find it. Not only is...