The tools we have seen so far all use statistical sampling. You often want to know more about the ordering of events so that you can see them and relate them to each other. Function tracing involves instrumenting the code with trace points that capture information about the event, and may include some or all of the following:
- Timestamp
- Context, such as the current PID
- Function parameters and return value
- Callstack
It is more intrusive than statistical profiling, and it can generate a large amount of data. The latter can be mitigated by applying filters when the sample is captured, and later on when viewing the trace.
I will cover two trace tools here: the kernel function tracers, Ftrace and LTTng.
