Backward Propagation
The previous section described how backward propagation in a computational graph is based on the chain rule. We will now cover how backward propagation works by taking operations, such as "+" and "x", as examples.
Backward Propagation in an Addition Node
First, let's consider backward propagation in an additional node. Here, we will look at backward propagation for the equation z = x + y. We can obtain the derivatives of z = x + y (analytically) as follows:
|
|
(5.5) |
As equation (5.5) shows, both 
and 
are 1. Therefore, we can represent them in a computational graph, as shown in the following diagram. In backward propagation, the derivative from the upper stream—
, in this example—is multiplied by 1 and passed downstream. In short, backward propagation in an addition node multiplies 1, so it only passes the input value to the...
