Inspecting SageMaker Debugger logs and results
In the previous recipe, we used SageMaker Debugger to help us detect a violation of a rule while the training job is running. In this recipe, we will use the smdebug library, the AWS CLI tool, and the awslogs command-line tool to inspect the files and logs generated by SageMaker Debugger. This will be very useful when debugging the results of SageMaker Debugger itself.
Sometimes, if we have a misconfigured setup with SageMaker Debugger, we may end up getting the NoIssuesFound result even if certain DebuggerRule conditions have been met already. That said, it is important that we know how to debug our setup and where to look for the files generated by SageMaker Debugger.
Getting ready
The following is the prerequisite for this recipe:
- This recipe continues from Identifying issues with SageMaker Debugger.
How to do it…
The following instructions show us the steps to inspect the files and logs generated...
