Search icon CANCEL
Arrow left icon
Explore Products
Best Sellers
New Releases
Books
Videos
Audiobooks
Learning Hub
Conferences
Free Learning
Arrow right icon
Linux Kernel Programming
Linux Kernel Programming

Linux Kernel Programming: A comprehensive guide to kernel internals, writing kernel modules, and kernel synchronization

Arrow left icon
Profile Icon Kaiwan N. Billimoria
Arrow right icon
Can$38.99 Can$55.99
Full star icon Full star icon Full star icon Full star icon Half star icon 4.6 (32 Ratings)
eBook Mar 2021 754 pages 1st Edition
eBook
Can$38.99 Can$55.99
Paperback
Can$69.99
Subscription
Free Trial
Arrow left icon
Profile Icon Kaiwan N. Billimoria
Arrow right icon
Can$38.99 Can$55.99
Full star icon Full star icon Full star icon Full star icon Half star icon 4.6 (32 Ratings)
eBook Mar 2021 754 pages 1st Edition
eBook
Can$38.99 Can$55.99
Paperback
Can$69.99
Subscription
Free Trial
eBook
Can$38.99 Can$55.99
Paperback
Can$69.99
Subscription
Free Trial

What do you get with eBook?

Product feature icon Instant access to your Digital eBook purchase
Product feature icon Download this book in EPUB and PDF formats
Product feature icon Access this title in our online reader with advanced features
Product feature icon DRM FREE - Read whenever, wherever and however you want
Product feature icon AI Assistant (beta) to help accelerate your learning
Table of content icon View table of contents Preview book icon Preview Book

Linux Kernel Programming

Kernel Workspace Setup

Hello, and welcome to this book on learning Linux kernel development. To get the most out of this book, it is very important that you first set up the workspace environment that we will be using throughout the book. This chapter will teach you exactly how to do this and get started.

We will install a recent Linux distribution, preferably as a Virtual Machine (VM), and set it up to include all the required software packages. We will also clone this book's code repository on GitHub, and learn about a few useful projects that will help along this journey.

The best way to learn something is to do so empirically – not taking anyone's word on anything at all, but trying it out and experiencing it for yourself. Hence, this book gives you many hands-on experiments and kernel code examples that you can and indeed must try out yourself; this will greatly aid in your making real progress and deeply learning and understanding various aspects of Linux kernel and driver development. So, let's begin!

This chapter will take us through the following topics, which will help us set up our environment:

  • Running Linux as a guest VM
  • Setting up the software – distribution and packages
  • A few additional useful projects

Technical requirements

You will need a modern desktop PC or laptop. Ubuntu Desktop specifies the following as "recommended system requirements" for the installation and usage of the distribution:

  • A 2 GHz dual core processor or better.
  • RAM:
    • Running on physical host: 2 GB or more system memory (more will certainly help).
    • Running as a guest VM: The host system should have at least 4 GB RAM (the more the better and the smoother the experience).
  • 25 GB of free hard drive space (I suggest more, at least double this).
  • Either a DVD drive or a USB port for the installer media (not required when setting up Ubuntu as a guest VM).
  • Internet access is definitely helpful and required at times.

As performing tasks such as building a Linux kernel from source is a very memory- and CPU-intensive process, I highly recommend that you try it out on a powerful Linux system with plenty of RAM and disk space to spare as well. It should be pretty obvious – the more RAM and CPU power the host system has, the better!

Like any seasoned kernel contributor, I would say that working on a native Linux system is best. However, for the purposes of this book, we cannot assume that you will always have a dedicated native Linux box available to you. So, we will assume that you are working on a Linux guest. Working within a guest VM also adds an additional layer of isolation and thus safety. 

Cloning our code repository: The complete source code for this book is freely available on GitHub at https://github.com/PacktPublishing/Linux-Kernel-Programming. You can clone and work on it by cloning the git tree, like so:

git clone https://github.com/PacktPublishing/Linux-Kernel-Programming.git

The source code is organized chapter-wise. Each chapter is represented as a directory – for example, ch1/ has the source code for this chapter. The root of the source tree has some code that is common to all chapters, such as the source files convenient.h, klib_llkd.c, as well as others.

For efficient code browsing, I would strongly recommend that you always index the code base with ctags(1) and/or cscope(1). For example, to set up the ctags index, just cd to the root of the source tree and type ctags -R .

Unless noted otherwise, the code output we show in the book is the output as seen on an x86-64 Ubuntu 18.04.3 LTS guest VM (running under Oracle VirtualBox 6.1). You should realize that due to (usually minor) distribution – and even within the same distributions but differing versions – differences, the output shown here may not perfectly match what you see on your Linux system.

Running Linux as a guest VM

As discussed previously, a practical and convenient alternative to using a native Linux system is to install and use the Linux distribution as a guest OS on a VM. It's key that you install a recent Linux distribution, preferably as a VM to be safe and avoid unpleasant data loss or other surprises. The fact is when working at the level of the kernel, abruptly crashing the system (and the data loss risks that arise thereof) is actually a commonplace occurrence. I recommend using Oracle VirtualBox 6.x (or the latest stable version) or other virtualization software, such as VMware Workstation.

Both of these are freely available. It's just that the code for this book has been tested on VirtualBox 6.1. Oracle VirtualBox is considered Open Source Software (OSS) and is licensed under the GPL v2 (the same as the Linux kernel). You can download it from https://www.virtualbox.org/wiki/Downloads. Its documentation can be found here: https://www.virtualbox.org/wiki/End-user_documentation.

The host system should be either MS Windows 10 or later (of course, even Windows 7 will work), a recent Linux distribution (for example, Ubuntu or Fedora), or macOS. So, let's get started by installing our Linux guest.

Left arrow icon Right arrow icon
Download code icon Download Code

Key benefits

  • Discover how to write kernel code using the Loadable Kernel Module framework
  • Explore industry-grade techniques to perform efficient memory allocation and data synchronization within the kernel
  • Understand the essentials of key internals topics such as kernel architecture, memory management, CPU scheduling, and kernel synchronization

Description

Linux Kernel Programming is a comprehensive introduction for those new to Linux kernel and module development. This easy-to-follow guide will have you up and running with writing kernel code in next-to-no time. This book uses the latest 5.4 Long-Term Support (LTS) Linux kernel, which will be maintained from November 2019 through to December 2025. By working with the 5.4 LTS kernel throughout the book, you can be confident that your knowledge will continue to be valid for years to come. You’ll start the journey by learning how to build the kernel from the source. Next, you’ll write your first kernel module using the powerful Loadable Kernel Module (LKM) framework. The following chapters will cover key kernel internals topics including Linux kernel architecture, memory management, and CPU scheduling. During the course of this book, you’ll delve into the fairly complex topic of concurrency within the kernel, understand the issues it can cause, and learn how they can be addressed with various locking technologies (mutexes, spinlocks, atomic, and refcount operators). You’ll also benefit from more advanced material on cache effects, a primer on lock-free techniques within the kernel, deadlock avoidance (with lockdep), and kernel lock debugging techniques. By the end of this kernel book, you’ll have a detailed understanding of the fundamentals of writing Linux kernel module code for real-world projects and products.

Who is this book for?

This book is for Linux programmers beginning to find their way with Linux kernel development. If you’re a Linux kernel and driver developer looking to overcome frequent and common kernel development issues, or understand kernel intervals, you’ll find plenty of useful information. You’ll need a solid foundation of Linux CLI and C programming before you can jump in.

What you will learn

  • Write high-quality modular kernel code (LKM framework) for 5.x kernels
  • Configure and build a kernel from source
  • Explore the Linux kernel architecture
  • Get to grips with key internals regarding memory management within the kernel
  • Understand and work with various dynamic kernel memory alloc/dealloc APIs
  • Discover key internals aspects regarding CPU scheduling within the kernel
  • Gain an understanding of kernel concurrency issues
  • Find out how to work with key kernel synchronization primitives

Product Details

Country selected
Publication date, Length, Edition, Language, ISBN-13
Publication date : Mar 19, 2021
Length: 754 pages
Edition : 1st
Language : English
ISBN-13 : 9781789955927
Vendor :
Linux Foundation
Languages :
Tools :

What do you get with eBook?

Product feature icon Instant access to your Digital eBook purchase
Product feature icon Download this book in EPUB and PDF formats
Product feature icon Access this title in our online reader with advanced features
Product feature icon DRM FREE - Read whenever, wherever and however you want
Product feature icon AI Assistant (beta) to help accelerate your learning

Product Details

Publication date : Mar 19, 2021
Length: 754 pages
Edition : 1st
Language : English
ISBN-13 : 9781789955927
Vendor :
Linux Foundation
Languages :
Tools :

Packt Subscriptions

See our plans and pricing
Modal Close icon
$19.99 billed monthly
Feature tick icon Unlimited access to Packt's library of 7,000+ practical books and videos
Feature tick icon Constantly refreshed with 50+ new titles a month
Feature tick icon Exclusive Early access to books as they're written
Feature tick icon Solve problems while you work with advanced search and reference features
Feature tick icon Offline reading on the mobile app
Feature tick icon Simple pricing, no contract
$199.99 billed annually
Feature tick icon Unlimited access to Packt's library of 7,000+ practical books and videos
Feature tick icon Constantly refreshed with 50+ new titles a month
Feature tick icon Exclusive Early access to books as they're written
Feature tick icon Solve problems while you work with advanced search and reference features
Feature tick icon Offline reading on the mobile app
Feature tick icon Choose a DRM-free eBook or Video every month to keep
Feature tick icon PLUS own as many other DRM-free eBooks or Videos as you like for just Can$6 each
Feature tick icon Exclusive print discounts
$279.99 billed in 18 months
Feature tick icon Unlimited access to Packt's library of 7,000+ practical books and videos
Feature tick icon Constantly refreshed with 50+ new titles a month
Feature tick icon Exclusive Early access to books as they're written
Feature tick icon Solve problems while you work with advanced search and reference features
Feature tick icon Offline reading on the mobile app
Feature tick icon Choose a DRM-free eBook or Video every month to keep
Feature tick icon PLUS own as many other DRM-free eBooks or Videos as you like for just Can$6 each
Feature tick icon Exclusive print discounts

Frequently bought together


Stars icon
Total Can$ 178.97 189.97 11.00 saved
Mastering Embedded Linux Programming
Can$58.99 Can$69.99
Linux Kernel Programming
Can$69.99
Linux Kernel Programming Part 2 - Char Device Drivers and Kernel Synchronization
Can$49.99
Total Can$ 178.97 189.97 11.00 saved Stars icon

Table of Contents

18 Chapters
Section 1: The Basics Chevron down icon Chevron up icon
Kernel Workspace Setup Chevron down icon Chevron up icon
Building the 5.x Linux Kernel from Source - Part 1 Chevron down icon Chevron up icon
Building the 5.x Linux Kernel from Source - Part 2 Chevron down icon Chevron up icon
Writing Your First Kernel Module - LKMs Part 1 Chevron down icon Chevron up icon
Writing Your First Kernel Module - LKMs Part 2 Chevron down icon Chevron up icon
Writing Your First Kernel Module - LKMs Part 2
Technical requirements
A "better" Makefile template for your kernel modules
Configuring a "debug" kernel
Cross-compiling a kernel module
Setting up the system for cross-compilation
Attempt 1 – setting the "special" environment variables
Attempt 2 – pointing the Makefile to the correct kernel source tree for the target
Attempt 3 – cross-compiling our kernel module
Attempt 4  – cross-compiling our kernel module
Gathering minimal system information
Being a bit more security-aware
Licensing kernel modules
Emulating "library-like" features for kernel modules
Performing library emulation via multiple source files
Understanding function and variable scope in a kernel module
Understanding module stacking
Trying out module stacking
Passing parameters to a kernel module
Declaring and using module parameters
Getting/setting module parameters after insertion
Module parameter data types and validation
Validating kernel module parameters
Overriding the module parameter's name
Hardware-related kernel parameters
Floating point not allowed in the kernel
Auto-loading modules on system boot
Module auto-loading – additional details
Kernel modules and security – an overview
Proc filesystem tunables affecting the system log
The cryptographic signing of kernel modules
Disabling kernel modules altogether
Coding style guidelines for kernel developers
Contributing to the mainline kernel
Getting started with contributing to the kernel
Summary
Questions
Further reading
Section 2: Understanding and Working with the Kernel Chevron down icon Chevron up icon
Kernel Internals Essentials - Processes and Threads Chevron down icon Chevron up icon
Memory Management Internals - Essentials Chevron down icon Chevron up icon
Kernel Memory Allocation for Module Authors - Part 1 Chevron down icon Chevron up icon
Kernel Memory Allocation for Module Authors - Part 1
Technical requirements
Introducing kernel memory allocators
Understanding and using the kernel page allocator (or BSA)
The fundamental workings of the page allocator
Freelist organization
The workings of the page allocator
Working through a few scenarios
The simplest case
A more complex case
The downfall case
Page allocator internals – a few more details
Learning how to use the page allocator APIs
Dealing with the GFP flags
Freeing pages with the page allocator
Writing a kernel module to demo using the page allocator APIs
Deploying our lowlevel_mem_lkm kernel module
The page allocator and internal fragmentation
The exact page allocator APIs
The GFP flags – digging deeper
Never sleep in interrupt or atomic contexts
Understanding and using the kernel slab allocator
The object caching idea
Learning how to use the slab allocator APIs
Allocating slab memory
Freeing slab memory
Data structures – a few design tips
The actual slab caches in use for kmalloc
Writing a kernel module to use the basic slab APIs
Size limitations of the kmalloc API
Testing the limits – memory allocation with a single call
Checking via the /proc/buddyinfo pseudo-file
Slab allocator – a few additional details
Using the kernel's resource-managed memory allocation APIs
Additional slab helper APIs
Control groups and memory
Caveats when using the slab allocator
Background details and conclusions
Testing slab allocation with ksize() – case 1
Testing slab allocation with ksize() – case 2
Interpreting the output from case 2
Graphing it
Slab layer implementations within the kernel
Summary
Questions
Further reading
Kernel Memory Allocation for Module Authors - Part 2 Chevron down icon Chevron up icon
Kernel Memory Allocation for Module Authors - Part 2
Technical requirements
Creating a custom slab cache
Creating and using a custom slab cache within a kernel module
Creating a custom slab cache
Using the new slab cache's memory
Destroying the custom cache
Custom slab – a demo kernel module
Understanding slab shrinkers
The slab allocator – pros and cons – a summation
Debugging at the slab layer
Debugging through slab poisoning
Trying it out – triggering a UAF bug
SLUB debug options at boot and runtime
Understanding and using the kernel vmalloc() API
Learning to use the vmalloc family of APIs
A brief note on memory allocations and demand paging
Friends of vmalloc()
Specifying the memory protections
Testing it – a quick Proof of Concept
Why make memory read-only?
The kmalloc() and vmalloc() APIs – a quick comparison
Memory allocation in the kernel – which APIs to use when
Visualizing the kernel memory allocation API set
Selecting an appropriate API for kernel memory allocation
A word on DMA and CMA
Stayin' alive – the OOM killer
Reclaiming memory – a kernel housekeeping task and OOM
Deliberately invoking the OOM killer
Invoking the OOM killer via Magic SysRq
Invoking the OOM killer with a crazy allocator program
Understanding the rationale behind the OOM killer
Case 1 – vm.overcommit set to 2, overcommit turned off
Case 2 – vm.overcommit set to 0, overcommit on, the default
Demand paging and OOM
Understanding the OOM score
Summary
Questions
Further reading
The CPU Scheduler - Part 1 Chevron down icon Chevron up icon
The CPU Scheduler - Part 2 Chevron down icon Chevron up icon
Section 3: Delving Deeper Chevron down icon Chevron up icon
Kernel Synchronization - Part 1 Chevron down icon Chevron up icon
Kernel Synchronization - Part 2 Chevron down icon Chevron up icon
About Packt Chevron down icon Chevron up icon
Other Books You May Enjoy Chevron down icon Chevron up icon

Customer reviews

Top Reviews
Rating distribution
Full star icon Full star icon Full star icon Full star icon Half star icon 4.6
(32 Ratings)
5 star 75%
4 star 15.6%
3 star 3.1%
2 star 3.1%
1 star 3.1%
Filter icon Filter
Top Reviews

Filter reviews by




Andreas Hecht May 31, 2021
Full star icon Full star icon Full star icon Full star icon Full star icon 5
This book covers the vast field that is Linux Kernel Programming with sufficient amount of detail needed for module/driver development. The book is a hands-on guide as each chapter features example code, designed to illustrate the presented topics, which you can try out for yourself. Along with the code come questions (with solutions), exercises and a very well composed 'Further Reading' list, all of which you can download from the books GitHub repository. The book starts with fundamentals such as 'Building the Linux Kernel from Source' and 'Writing Your First Kernel Module' and leads to advanced topics such as memory management, scheduling and synchronization/concurrency. Thanks to the easy to follow instructions, this book is for anyone with a minimal amount of experience with Linux, C and embedded-/low-level-programming. Aside from the sound technical expertise this book brings to the table, it is written in a style that is inspiring and motivational. This book is actually fun to read and I would recommend it to anyone who would like to learn the skills and knowledge needed to become a Linux Kernel developer.
Amazon Verified review Amazon
David Conteh May 19, 2022
Full star icon Full star icon Full star icon Full star icon Full star icon 5
Great book. Well written and easy to follow.
Amazon Verified review Amazon
Swaran May 30, 2021
Full star icon Full star icon Full star icon Full star icon Full star icon 5
I was trying to find out a book with latest linux kernel development. Most of the available books are outdated. Finally found this book. This book provide the study material for which I was waiting since long. Book is written in clear and precise manner and guide the reader for "learning by doing" methodology.Complete book is divide into three sections1. First section explains about setting up the development environment, how to build linux kernel(of-course kernel 5.x) from source, how to write the kernel module2. Second section give in-depth understanding about kernel internals, processes, threads, memory management internals, kernel memory allocation, and deep understanding of CPU Schedulers.3. Third section goes even more deeper and explain about kernel synchronisation- critical sections, mutex and spinlock, debugging of locks in kernel.
Amazon Verified review Amazon
Shashank Apr 06, 2021
Full star icon Full star icon Full star icon Full star icon Full star icon 5
I met this guy and he is the most genuine person who has knowledge on kernel in depth. Read his book and helps me to brush the concepts and gives more charity on various topics such as memory management, cpu scheduling, interrupts, task structure, driver related concepts and many things. you will find the latest information as compare to other LDD books.definetly recommend this book to all who wants to deep dive into coding level, kernel concepts.
Amazon Verified review Amazon
Nayana Rao Mar 23, 2021
Full star icon Full star icon Full star icon Full star icon Full star icon 5
If you are looking for a comprehensive guide to Linux kernel and it’s configuration for a custom platform, I highly recommend this book. The book is very detailed and comes with the Github source base with a lot of sample codes to assist you.Disclaimer: I have helped with the technical review of this book.
Amazon Verified review Amazon
Get free access to Packt library with over 7500+ books and video courses for 7 days!
Start Free Trial

FAQs

How do I buy and download an eBook? Chevron down icon Chevron up icon

Where there is an eBook version of a title available, you can buy it from the book details for that title. Add either the standalone eBook or the eBook and print book bundle to your shopping cart. Your eBook will show in your cart as a product on its own. After completing checkout and payment in the normal way, you will receive your receipt on the screen containing a link to a personalised PDF download file. This link will remain active for 30 days. You can download backup copies of the file by logging in to your account at any time.

If you already have Adobe reader installed, then clicking on the link will download and open the PDF file directly. If you don't, then save the PDF file on your machine and download the Reader to view it.

Please Note: Packt eBooks are non-returnable and non-refundable.

Packt eBook and Licensing When you buy an eBook from Packt Publishing, completing your purchase means you accept the terms of our licence agreement. Please read the full text of the agreement. In it we have tried to balance the need for the ebook to be usable for you the reader with our needs to protect the rights of us as Publishers and of our authors. In summary, the agreement says:

  • You may make copies of your eBook for your own use onto any machine
  • You may not pass copies of the eBook on to anyone else
How can I make a purchase on your website? Chevron down icon Chevron up icon

If you want to purchase a video course, eBook or Bundle (Print+eBook) please follow below steps:

  1. Register on our website using your email address and the password.
  2. Search for the title by name or ISBN using the search option.
  3. Select the title you want to purchase.
  4. Choose the format you wish to purchase the title in; if you order the Print Book, you get a free eBook copy of the same title. 
  5. Proceed with the checkout process (payment to be made using Credit Card, Debit Cart, or PayPal)
Where can I access support around an eBook? Chevron down icon Chevron up icon
  • If you experience a problem with using or installing Adobe Reader, the contact Adobe directly.
  • To view the errata for the book, see www.packtpub.com/support and view the pages for the title you have.
  • To view your account details or to download a new copy of the book go to www.packtpub.com/account
  • To contact us directly if a problem is not resolved, use www.packtpub.com/contact-us
What eBook formats do Packt support? Chevron down icon Chevron up icon

Our eBooks are currently available in a variety of formats such as PDF and ePubs. In the future, this may well change with trends and development in technology, but please note that our PDFs are not Adobe eBook Reader format, which has greater restrictions on security.

You will need to use Adobe Reader v9 or later in order to read Packt's PDF eBooks.

What are the benefits of eBooks? Chevron down icon Chevron up icon
  • You can get the information you need immediately
  • You can easily take them with you on a laptop
  • You can download them an unlimited number of times
  • You can print them out
  • They are copy-paste enabled
  • They are searchable
  • There is no password protection
  • They are lower price than print
  • They save resources and space
What is an eBook? Chevron down icon Chevron up icon

Packt eBooks are a complete electronic version of the print edition, available in PDF and ePub formats. Every piece of content down to the page numbering is the same. Because we save the costs of printing and shipping the book to you, we are able to offer eBooks at a lower cost than print editions.

When you have purchased an eBook, simply login to your account and click on the link in Your Download Area. We recommend you saving the file to your hard drive before opening it.

For optimal viewing of our eBooks, we recommend you download and install the free Adobe Reader version 9.