Training multi-layer perceptrons

In this section, we will discuss how to train a multi-layer perceptron. Recall from Chapter 5, From Simple Linear Regression to Multiple Linear Regression that we can use gradient descent to minimize a real-valued function, C,  of many variables. Assume that C is a function of two variables v₁ and v₂. To understand how to change the variables to minimize C, we need a small change in the variables to produce a small change in the output. We will represent a change in the value of v₁ with Δv₁, a change in the value of v₂ with Δv₂, and a change in the value of C with ΔC. The relation between ΔC and changes to variables is given by:

∂C/∂v₁ is the partial derivative of C with respect to v₁. For convenience, we will represent Δv1 and...