Observability in action

There are several issues in our server application. First issue reproduces sporadically when you hit the frontend at http://localhost:5051/single/hello several times. You might notice that some requests take longer than others. If we look at the duration metrics or Jaeger’s duration view, we’ll see something similar to Figure 10.6:

Figure 10.6 – Duration view in Jaeger

We see that most of the calls are fast (around 100 ms), but there is one that takes longer than a second. If we click on it, Jaeger will open the corresponding trace, like the one shown in Figure 10.7:

Figure 10.7 – Long trace with errors

Apparently, there were three attempts to send the message – the first two were not successful, but the third one succeeded. So retries are the source of long latency. We can investigate the error by expanding the exception event – we’ll see a full stack...