Search icon CANCEL
Subscription
0
Cart icon
Your Cart (0 item)
Close icon
You have no products in your basket yet
Arrow left icon
Explore Products
Best Sellers
New Releases
Books
Videos
Audiobooks
Learning Hub
Conferences
Free Learning
Arrow right icon
Arrow up icon
GO TO TOP
Architecture and Design of the Linux Storage Stack

You're reading from   Architecture and Design of the Linux Storage Stack Gain a deep understanding of the Linux storage landscape and its well-coordinated layers

Arrow left icon
Product type Paperback
Published in Jul 2023
Publisher Packt
ISBN-13 9781837639960
Length 246 pages
Edition 1st Edition
Tools
Concepts
Arrow right icon
Author (1):
Arrow left icon
Muhammad Umer Muhammad Umer
Author Profile Icon Muhammad Umer
Muhammad Umer
Arrow right icon
View More author details
Toc

Table of Contents (18) Chapters Close

Preface 1. Part 1: Diving into the Virtual Filesystem
2. Chapter 1: Where It All Starts From – The Virtual Filesystem FREE CHAPTER 3. Chapter 2: Explaining the Data Structures in a VFS 4. Chapter 3: Exploring the Actual Filesystems Under the VFS 5. Part 2: Navigating Through the Block Layer
6. Chapter 4: Understanding the Block Layer, Block Devices, and Data Structures 7. Chapter 5: Understanding the Block Layer, Multi-Queue, and Device Mapper 8. Chapter 6: Understanding I/O Handling and Scheduling in the Block Layer 9. Part 3: Descending into the Physical Layer
10. Chapter 7: The SCSI Subsystem 11. Chapter 8: Illustrating the Layout of Physical Media 12. Part 4: Analyzing and Troubleshooting Storage Performance
13. Chapter 9: Analyzing Physical Storage Performance 14. Chapter 10: Analyzing Filesystems and the Block Layer 15. Chapter 11: Tuning the I/O Stack 16. Index 17. Other Books You May Enjoy

Explaining the Everything is a file philosophy

In Linux, all of the following are considered files:

  • Directories
  • Disk drives and their partitions
  • Sockets
  • Pipes
  • CD-ROM

The phrase everything is a file implies that all the preceding entities in Linux are represented by file descriptors, abstracted over the VFS. You could also say that everything has a file descriptor, but let’s not indulge in that debate.

The everything is a file ideology that characterizes the architecture of a Linux system is also implemented courtesy of the VFS. Earlier, we defined pseudo filesystems as filesystems that generate their content on the fly. These filesystems are also referred to as VFSes and play a major role in implementing this concept.

You can retrieve the list of filesystems currently registered with the kernel through the procfs pseudo filesystem. When seeing this list, note nodev in the first column against some filesystems. nodev indicates that this is a pseudo filesystem and is not backed by a block device. Filesystems such as Ext2, 3, and 4 are created on a block device; hence, they do not have the nodev entry in the first column:

cat /proc/filesystems
[root@linuxbox ~]# cat /proc/filesystems
nodev   sysfs
nodev   tmpfs
nodev   bdev
nodev   proc
nodev   cgroup
nodev   cgroup2
nodev   cpuset
nodev   devtmpfs
nodev   configfs
nodev   debugfs
nodev   tracefs
nodev   securityfs
nodev   sockfs
nodev   bpf
nodev   pipefs
nodev   ramfs

[The rest of the code is skipped for brevity.]

You can also use the mount command to find out about the currently mounted pseudo filesystems in your system:

mount | grep -v sd | grep -ivE ":/|mapper"
[root@linuxbox ~]# mount | grep -v sd | grep -ivE ":/|mapper"
sysfs on /sys type sysfs (rw,nosuid,nodev,noexec,relatime)
proc on /proc type proc (rw,nosuid,nodev,noexec,relatime)
devtmpfs on /dev type devtmpfs (rw,nosuid,size=1993552k,nr_inodes=498388,mode=755)
securityfs on /sys/kernel/security type securityfs (rw,nosuid,nodev,noexec,relatime)
tmpfs on /dev/shm type tmpfs (rw,nosuid,nodev)
devpts on /dev/pts type devpts (rw,nosuid,noexec,relatime,gid=5,mode=620,ptmxmode=000)
tmpfs on /run type tmpfs (rw,nosuid,nodev,mode=755)
tmpfs on /sys/fs/cgroup type tmpfs (ro,nosuid,nodev,noexec,mode=755)
cgroup on /sys/fs/cgroup/systemd type cgroup (rw,nosuid,nodev,noexec,relatime,xattr,release_agent=/usr/lib/systemd/systemd-cgroups-agent,name=systemd)
pstore on /sys/fs/pstore type pstore (rw,nosuid,nodev,noexec,relatime)
efivarfs on /sys/firmware/efi/efivars type efivarfs (rw,nosuid,nodev,noexec,relatime)

[The rest of the code is skipped for brevity.]

Let’s take a tour of the /proc directory. You’ll see a long list of numbered directories; these numbers represent the IDs of all the processes that currently run on your system:

[root@linuxbox ~]# ls /proc/
1    1116   1228072  1235  1534  196  216  30   54   6  631  668  810      ioports     scsi
10   1121   1228220  1243  1535  197  217  32   55   600  632  670  9      irq         self
1038  1125  1228371  1264  1536  198  218  345  56   602  
633  673  905      kallsyms     slabinfo
1039  1127  1228376  13    1537  199  219  347  570  603  
634  675  91       kcore      softirqs
1040  1197  1228378  14    1538  2    22   348  573  605  635  677  947      keys       stat
1041  12    1228379  1442  16    20   220  37   574  607  
636  679  acpi     key-users  swaps
1042  1205  1228385  1443  1604  200  221  38   576  609  
637  681  buddyinfo  kmsg       sys
1043  1213  1228386  1444  1611  201  222  39   577  610  638  684  bus      kpagecgroup  sysrq-

[The rest of the code is skipped for brevity.]

The procfs filesystem offers us a glimpse into the running state of the kernel. The content in /proc is generated when we want to view this information. This information is not persistently present on your disk drive. This all happens in memory. As you can see from the ls command, the size of /proc on disk is zero bytes:

[root@linuxbox ~]# ls -ld /proc/
dr-xr-xr-x 292 root root 0 Sep 20 00:41 /proc/
[root@linuxbox ~]#

/proc provides an on-the-spot view of the processes running on the system. Consider the /proc/cpuinfo file. This file displays the processor-related information for your system. If we check this file, it will be shown as empty:

[root@linuxbox ~]# ls -l /proc/cpuinfo
-r--r--r-- 1 root root 0 Nov  5 02:02 /proc/cpuinfo
[root@linuxbox ~]#
[root@linuxbox ~]# file /proc/cpuinfo
/proc/cpuinfo: empty
[root@linuxbox ~]#

However, when the file contents are viewed through cat, they show a lot of information:

[root@linuxbox ~]# cat /proc/cpuinfo
processor       : 0
vendor_id       : GenuineIntel
cpu family      : 6
model           : 79
model name      : Intel(R) Xeon(R) CPU E5-2683 v4 @ 2.10GHz
stepping        : 1
microcode       : 0xb00003e
cpu MHz         : 2099.998
cache size      : 40960 KB
physical id     : 0
siblings        : 1
core id         : 0
cpu cores       : 1
apicid          : 0
initial apicid  : 0
fpu             : yes
fpu_exception   : yes
cpuid level     : 20
wp              : yes

[The rest of the code is skipped for brevity.]

Linux abstracts all entities such as processes, directories, network sockets, and storage devices into the VFS. Through the VFS, we can retrieve information from the kernel. Most Linux distributions offer a variety of tools for monitoring the consumption of storage, compute, and memory resources. All these tools gather stats for various metrics through the data available in procfs. For instance, the mpstat command, which provides stats about all the processors in a system, retrieves data from the /proc/stat file. It then presents this data in a human-readable format for a better understanding:

[root@linuxbox ~]# cat /proc/stat
cpu  5441359 345061 4902126 1576734730 46546 1375926 942018 0 0 0
cpu0 1276258 81287 1176897 394542528 13159 255659 280236 0 0 0
cpu1 1455759 126524 1299970 394192241 13392 314865 178446 0 0 0
cpu2 1445048 126489 1319450 394145153 12496 318550 186289 0 0 0
cpu3 1264293 10760 1105807 393854806 7498 486850 297045 0 0 0

[The rest of the code is skipped for brevity.]

If we use the strace utility on the mpstat command, it will show that under the hood, mpstat uses the /proc/stat file to display processor stats:

strace mpstat 2>&1 |grep "/proc/stat"
[root@linuxbox ~]# strace mpstat 2>&1 |grep "/proc/stat"
openat(AT_FDCWD, "/proc/stat", O_RDONLY) = 3
[root@linuxbox ~]#

Similarly, popular commands such as top, ps, and free gather memory-related information from the /proc/meminfo file:

[root@linuxbox ~]# strace free -h 2>&1 |grep meminfo
openat(AT_FDCWD, "/proc/meminfo", O_RDONLY) = 3
[root@linuxbox ~]#

Similar to /proc, another commonly used pseudo filesystem is sysfs, which is mounted at /sys. The sysfs filesystem mostly contains information about hardware devices on your system. For example, to find information about the disk drive in your system, such as its model, you can issue the following command:

cat /sys/block/sda/device/model
[root@linuxbox ~]# cat /sys/block/sda/device/model
SAMSUNG MZMTE512
[root@linuxbox ~]#

Even LEDs on a keyboard have a corresponding file in /sys:

[root@linuxbox ~]# ls /sys/class/leds
ath9k-phy0  input4::capslock  input4::numlock  input4::scrolllock
[root@linuxbox ~]#

The everything is a file philosophy is one of the defining features of the Linux kernel. It signifies that everything in a system, including regular text files, directories, and devices, can be abstracted over the VFS layer in the kernel. As a result, all these entities can be represented as file-like objects through the VFS layer. There are several pseudo filesystems in Linux that contain information about the different kernel subsystems. The content of these pseudo filesystems is only present in memory and generated dynamically.

You have been reading a chapter from
Architecture and Design of the Linux Storage Stack
Published in: Jul 2023
Publisher: Packt
ISBN-13: 9781837639960
Register for a free Packt account to unlock a world of extra content!
A free Packt account unlocks extra newsletters, articles, discounted offers, and much more. Start advancing your knowledge today.
Unlock this book and the full library FREE for 7 days
Get unlimited access to 7000+ expert-authored eBooks and videos courses covering every tech area you can think of
Renews at $19.99/month. Cancel anytime
Banner background image