When performing an operation, for example training a neural network, or the sum of two integers, TensorFlow internally represent, its computation using a data flow graph (or computational graph).
This is a directed graph consisting of the following:
- A set of nodes, each one representing an operation
- A set of directed arcs, each one representing the data on which the operations are performed
TensorFlow has two types of edge:
- Normal: They are only carriers of data structures, between the nodes. The output of one operation (from one node) becomes the input for another operation. The edge connecting two nodes carry the values.
- Special: This edge doesn't carry values. It represents a control dependency between two nodes A and B. It means that the node B will be executed only if the operation in A will be ended before the relationship between operations on the data.
The TensorFlow implementation...
