Summary
In previous chapters, we explored telemetry auto-collection, and now we have learned how to customize this telemetry. We learned about different sampling approaches – head-based and tail-based. Head-based sampling makes the decision to record a trace (or span) when it starts with a certain probability. Children can follow parent decisions and then traces are always complete, but it’s not possible to control the volume of traces on individual services. To overcome this, downstream services can configure different rates and use consistent sampling to maximize the number of complete traces. Some traces become partial but are still useful for monitoring individual services or groups of them.
Probability sampling captures a percentage of all traces and is great to mitigate performance overhead. If you need predictable costs, you should consider rate-based sampling. It’s implemented in the OpenTelemetry Collector or by observability vendors. The OpenTelemetry...
