Gradient calculations for gradient descent algorithms

One of TenorFlow's great strengths is its ability to automatically compute gradients for use in gradient descent algorithms, which, of course, are a vital part of most machine learning models. TensorFlow offers a number of methods for gradient calculations.

There are four ways to automatically compute gradients when eager execution is enabled (they also work in graph mode):

1. tf.GradientTape: Context records computations so that you can call tf.gradient() to get the gradients of any tensor computed while recording with respect to any trainable variable
2. tfe.gradients_function(): Takes a function (say f()) and returns a gradient function (say fg()) that can compute the gradients of the outputs of f() with respect to the parameters of f() or a subset of them
3. tfe.implicit_gradients(): This is very similar, but fg() computes...