Tracing events
The tools we have seen so far all use statistical sampling. You often want to know more about the ordering of events so that you can see them and relate them to each other. Function tracing involves instrumenting the code with tracepoints that capture information about the event, and may include some or all of the following:
- A timestamp
- Context, such as the current PID
- Function parameters and return values
- A callstack
It is more intrusive than statistical profiling and it can generate a large amount of data. The latter problem can be mitigated by applying filters when the sample is captured and later on when viewing the trace.
I will cover three trace tools here: the kernel function tracers Ftrace, LTTng, and BPF.