Deep Learning Fundamentals

In this chapter, we will introduce deep learning(DL) and deep neural networks (DNNs), that is, neural networks with multiple hidden layers. You may wonder what the point of using more than one hidden layer is, given the universal approximation theorem. This is in no way a naive question, and for a long time neural networks were used in that way. Without going into too much detail, one reason is that approximating a complex function might require a huge number of neurons in the hidden layer, making it impractical to use. There is also another, more important, reason for using deep networks, which is not directly related to the number of hidden layers, but to the level of learning. A deep network does not simply learn to predict output Y given input X; it also understands basic features of the input. It's able to learn abstractions of features of...