This section focuses mostly on practical advice that can be directly implemented in your code. The TensorFlow team has provided a large set of tools that can be utilized to improve your performance. These techniques are constantly being updated to achieve better results. I strongly recommend watching TensorFlow's video on training performance from the 2018 TensorFlow conference (https://www.youtube.com/watch?v=SxOsJPaxHME). This video is accompanied by nicely aggregated documentation, which is also a must-read (https://www.tensorflow.org/performance/).
Now, let's dive into more details around what you can do to achieve faster and more reliable training.
Let's first start with an illustration from TensorFlow that presents the general steps of training a neural network. You can divide this process into three...
