Summary

In this chapter, we explored messaging instrumentation. We started with messaging specifics and the new challenges they bring to observability. We briefly looked into OpenTelemetry messaging semantic conventions and then dived into producer instrumentation. The producer is responsible for injecting trace context into messages and instrumenting publish operations so that it’s possible to trace each independent flow on consumers.

Then, we instrumented the consumer with metrics and traces. We learned how to measure consumer health using queue size and lag and explored the instrumentation options for batching scenarios. Finally, we saw how we can use instrumentation to detect and investigate common messaging issues.

With this, you’re prepared to instrument common messaging patterns and can start designing and tuning instrumentation for advanced streaming scenarios.

In the next chapter, we’re going to design a comprehensive observability store for databases...