Learning from Data

The essential characteristic of a neural network is its ability to learn from data. Training from data means that weight parameter values can be automatically determined. If you have to determine all the parameters manually, it is quite hard work. For example, for a sample perceptron, as shown in Chapter 2, Perceptrons, we determined the parameter values manually while looking at the truth table. There are as few as three parameters. However, in an actual neural network, the number of parameters can range between thousands and tens of thousands. For deep learning with more layers, the number of parameters may reach hundreds of millions. It is almost impossible to determine them manually. This chapter describes neural network training, or how to determine parameter values from data, and implements a model that learns handwritten digits from the MNIST dataset with Python.

Note

For a linearly separable problem, a perceptron can learn automatically from data. That...