The following are a few tools worth mentioning with regard to capturing and analyzing system latencies (and more):
- You can learn how to set up and use the powerful Linux Tracing Toolkit – next generation (LTTng) toolset to record traces of the system in action. I highly recommend using the superb Trace Compass GUI to analyze it. In fact, in the companion guide Linux Kernel Programming - Chapter 1, Kernel Workspace Setup, in the Linux Tracing Toolkit next generation (LTTng) section, we showed an interesting screenshot (Figure 1.9) of the Trace Compass GUI being used to display and analyze IRQ lines 1 and 130 (the interrupt lines for the i8042 and Wi-Fi chipset on my native x86_64 system, respectively).
- You can also try using the latencytop tool to determine which kernel ops what user space threads are blocking on. You will have to turn on CONFIG_LATENCYTOP in the kernel config to do this.
- Besides latency...
