Instrumenting the application with OpenTelemetry and Prometheus

Our application has already been set up with a logger, but we need traces and metrics to achieve observability. The OpenTelemetry project aims to support all three (logging, traces, and metrics) in the Go SDK but at the time of writing this book and version v1.10, only tracing is stable. So, we will leave our logger in place and interact directly with Prometheus for our metrics. We will begin with OpenTelemetry and distributed tracing.

Adding distributed tracing to the application

Getting started with OpenTelemetry is very easy; first, we will need to create a connection to the collector. In our application, we will have one running and available at the default port. The monolith or microservices will use the following environment variables to configure themselves:

OTEL_SERVICE_NAME: mallbots
OTEL_EXPORTER_OTLP_ENDPOINT: http://collector:4317

The OpenTelemetry SDK we will use will look for specific variables...