Chain Rule

Forward propagation in a computational graph propagates the calculation result in the forward direction from left to right. These calculations seem natural because they are usually conducted. On the other hand, in backward propagation, a "local derivative" is propagated in the backward direction from right to left. The principle that propagates the "local derivative" is based on the chain rule. Let's look at the chain rule and clarify how it corresponds to backward propagation in a computational graph.

Backward Propagation in a Computational Graph

We will now look at an example of backward propagation using a computational graph. Let's assume that a calculation, y = f (x), exists. The following diagram shows the backward propagation of this calculation:

Figure 5.6: Backward propagation in a computational graph – the local derivative is multiplied in the backward direction

As shown in the preceding diagram...