Best practices for debugging and troubleshooting Secrets
When a Secret is wrong, it can affect us in ways that are not immediately visible. A top-down approach can be used where we start by checking the actual application that is affected. Eventually, we will reach a point where the logs will point to the misconfigured Secret. Once we reach the Secret, we should identify whether the Secret has been applied correctly or whether it is a wrong Secret.
We could make a checklist when evaluating the Secret:
- Ensure the existence of the Secret.
- Check the value of the Secret.
- Decode the Secret and see whether it is the desired one.
- Use MD5 hashing.
- Avoid downloading Secrets locally.
The next thing to check is whether the application of the Secret is wrong. Imagine a scenario where a Secret is mounted on a Pod of a deployment incorrectly. There is the option to continuously try to change the deployment and eventually figure out along the way what is wrong. However...