Debugging with the TensorFlow debugger (tfdbg)

The TensorFlow debugger (tfdbg) works the same way at a high level as other popular debuggers, such as pdb and gdb. To use a debugger, the process is generally as follows:

1. Set the breakpoints in the code at locations where you want to break and inspect the variables
2. Run the code in debug mode
3. When the code breaks at a breakpoint, inspect it and then move on to next step

Some debuggers also allow you to interactively watch the variables while the code is executing, not just at the breakpoint:

1. In order to use tfdbg, first import the required modules and wrap the session inside a debugger wrapper:

from tensorflow.python import debug as tfd

with tfd.LocalCLIDebugWrapperSession(tf.Session()) as tfs:

2. Next, attach a filter to the session object. Attaching a filter is the same as setting a breakpoint in other debuggers. For example, the following code attaches a tfdbg.has_inf_or_nan filter which breaks if any of the intermediate tensors have nan or inf values...