We'll start with a simple example to illustrate graph creation and optimization:
def compute(a, b, c):
d = a * b + c
e = a * b * c
return d, e
Assuming a, b, and c are Tensor matrices, this code computes two new values: d and e. Using eager execution, TensorFlow would compute the value for d and then compute the value for e.
Using lazy execution, TensorFlow would create a graph of operations. Before running the graph to get the result, a graph optimizer would be run. To avoid computing a * b twice, the optimizer would cache the result and reuse it when necessary. For more complex operations, the optimizer could enable parallelism to make computation faster. Both techniques are important when running large and complex models.
As we saw, running in eager mode implies that every operation is run when defined. Therefore, such optimizations cannot be applied. Thankfully, TensorFlow includes a module to work around this—TensorFlow...
