Tracing VFS functions

There are quite a few tracing mechanisms available in Linux that can offer a glance at how things work under the hood. One of them is the BPF Compiler Collection (BCC) tools. These tools offer a wide range of scripts that can record events for different subsystems in the kernel. You can install these tools for your operating system by following the instructions in the Technical requirements section. For now, we’re just going to use one of the programs from this toolkit, called funccount. As the name suggests, funccount counts the number of function calls:

root@linuxbox:~# funccount --help
usage: funccount [-h] [-p PID] [-i INTERVAL] [-d DURATION] [-T] [-r] [-D]
                 [-c CPU]
                 pattern
Count functions, tracepoints, and USDT probes

Just to test and verify our understanding of what we stated earlier, we’re going to run a simple copy process in the background and use the funccount program to trace the VFS functions that are invoked as a result of the cp command. As we’re going to count the VFS calls for the cp process only, we need to use the -p flag to specify a process ID. The vfs_* parameter will trace all the VFS functions for the process. You’ll see that the vfs_read () and vfs_write () functions are invoked by the cp process. The COUNT column specifies the number of times the function was called:

funccount -p process_ID 'vfs_*'
[root@linuxbox ~]# nohup cp myfile /tmp/myfile &
[1] 1228433
[root@linuxbox ~]# nohup: ignoring input and appending output to 'nohup.out'
[root@linuxbox ~]#
[root@linuxbox ~]# funccount -p 1228433 "vfs_*"
Tracing 66 functions for "b'vfs_*'"... Hit Ctrl-C to end.
^C
FUNC                                    COUNT
b'vfs_read'                             28015
b'vfs_write'                            28510
Detaching...
[root@linuxbox ~]#

Let’s run this again and see what system calls are used when doing a simple copy operation. As expected, the most frequently used system calls when doing cp are read and write:

funccount 't:syscalls:sys_enter_*' -p process_ID
[root@linuxbox ~]# nohup cp myfile /tmp/myfile &
[1] 1228433
[root@linuxbox ~]# nohup: ignoring input and appending output to 'nohup.out'
[root@linuxbox ~]#
[root@linuxbox ~]# /usr/share/bcc/tools/funccount -p 1228433 "vfs_*"
Tracing 66 functions for "b'vfs_*'"... Hit Ctrl-C to end.
^C
FUNC                                    COUNT
b'vfs_read'                             28015
b'vfs_write'                            28510
Detaching...
[root@linuxbox ~]#

Let’s summarize what we covered in this section. Linux offers support for a wide range of filesystems, and the VFS layer in the kernel ensures that this can be achieved without any hassle. The VFS provides a standardized way for end user processes to interact with the different filesystems. This standardization is achieved by implementing a common file mode. The VFS defines several virtual functions for common file operations. As a result of this approach, applications can universally perform regular file operations. When a process generates a system call, the VFS will redirect these calls to the appropriate function of the filesystem.