Book Image

Linux Kernel Programming - Second Edition

By : Kaiwan N. Billimoria
Book Image

Linux Kernel Programming - Second Edition

By: Kaiwan N. Billimoria

Overview of this book

The 2nd Edition of Linux Kernel Programming is an updated, comprehensive guide for new programmers to the Linux kernel. This book uses the recent 6.1 Long-Term Support (LTS) Linux kernel series, which will be maintained until Dec 2026, and also delves into its many new features. Further, the Civil Infrastructure Project has pledged to maintain and support this 6.1 Super LTS (SLTS) kernel right until August 2033, keeping this book valid for years to come! You’ll begin this exciting journey by learning how to build the kernel from source. In a step by step manner, you will then learn how to write your first kernel module by leveraging the kernel’s powerful Loadable Kernel Module (LKM) framework. With this foundation, you will delve into key kernel internals topics including Linux kernel architecture, memory management, and CPU (task) scheduling. You’ll finish with understanding the deep issues of concurrency, and gain insight into how they can be addressed with various synchronization/locking technologies (e.g., mutexes, spinlocks, atomic/refcount operators, rw-spinlocks and even lock-free technologies such as per-CPU and RCU). By the end of this book, you’ll have a much better understanding of the fundamentals of writing the Linux kernel and kernel module code that can straight away be used in real-world projects and products.

Related Content you might be interested in

Current Title:

Small book image
Linux Kernel Programming - Second Edition
Kaiwan N. Billimoria
Feb, 2024 | 826 pages
Small book image
Linux Kernel Programming Part 2 - Char Device Drivers and Kernel Synchronization
Kaiwan N Billimoria
Mar, 2021 | 452 pages
Small book image
Linux Kernel Debugging
Kaiwan N Billimoria
Aug, 2022 | 638 pages
Small book image
Hands-On System Programming with Linux
Tigran Aivazian | Kaiwan N Billimoria
Oct, 2018 | 794 pages
Table of Contents (16 chapters)
Preface
Preface
Who this book is for
What’s been added in the second edition?
What this book covers
Get in touch
1
Linux Kernel Programming – A Quick Introduction
Linux Kernel Programming – A Quick Introduction
Kernel workspace setup
Technical requirements
Cloning this book’s code repository
Free Chapter
2
Building the 6.x Linux Kernel from Source – Part 1
Building the 6.x Linux Kernel from Source – Part 1
Technical requirements
Preliminaries for the kernel build
Steps to build the kernel from source
Step 1 – Obtaining a Linux kernel source tree
Step 2 – Extracting the kernel source tree
Step 3 – Configuring the Linux kernel
Customizing the kernel menu, Kconfig, and adding our own menu item
Summary
Exercise
Questions
Further reading
3
Building the 6.x Linux Kernel from Source – Part 2
Building the 6.x Linux Kernel from Source – Part 2
Technical requirements
Step 4 – building the kernel image and modules
Step 5 – installing the kernel modules
Step 6 – generating the initramfs image and bootloader setup
Understanding the initramfs framework
Step 7 – customizing the GRUB bootloader
Verifying our new kernel’s configuration
Kernel build for the Raspberry Pi
Miscellaneous tips on the kernel build
Summary
Questions
Further reading
4
Writing Your First Kernel Module – Part 1
Writing Your First Kernel Module – Part 1
Technical requirements
Understanding the kernel architecture – part 1
Exploring LKMs
Writing our very first kernel module
Common operations on kernel modules
Understanding kernel logging and printk
Understanding the basics of a kernel module Makefile
Summary
Questions
Further reading
5
Writing Your First Kernel Module – Part 2
Writing Your First Kernel Module – Part 2
Technical requirements
A “better” Makefile template for your kernel modules
Cross-compiling a kernel module
Gathering minimal system information
Licensing kernel modules
Emulating “library-like” features for kernel modules
Passing parameters to a kernel module
Floating point not allowed in the kernel
Auto-loading modules on system boot
Kernel modules and security – an overview
Coding style guidelines for kernel developers
Contributing to the mainline kernel
Summary
Questions
Further reading
6
Kernel Internals Essentials – Processes and Threads
Kernel Internals Essentials – Processes and Threads
Technical requirements
Understanding process and interrupt contexts
Understanding the basics of the process Virtual Address Space (VAS)
Organizing processes, threads, and their stacks – user and kernel space
Understanding and accessing the kernel task structure
Working with the task structure via ‘current’
Iterating over the kernel’s task lists
Summary
Questions
Further reading
7
Memory Management Internals – Essentials
Memory Management Internals – Essentials
Technical requirements
Understanding the VM split
Examining the process VAS
Examining the kernel VAS
Randomizing the memory layout – KASLR
Understanding physical memory organization
Summary
Questions
Further reading
8
Kernel Memory Allocation for Module Authors – Part 1
Kernel Memory Allocation for Module Authors – Part 1
Technical requirements
Introducing kernel memory allocators
Understanding and using the kernel page allocator (or BSA)
Understanding and using the kernel slab allocator
Size limitations of the kmalloc API
Slab allocator – a few additional details
Caveats when using the slab allocator
Summary
Questions
Further reading
9
Kernel Memory Allocation for Module Authors – Part 2
Kernel Memory Allocation for Module Authors – Part 2
Technical requirements
Creating a custom slab cache
Debugging kernel memory issues – a quick mention
Understanding and using the kernel vmalloc() API
Memory allocation in the kernel – which APIs to use when
Memory reclaim – a key kernel housekeeping task
Stayin’ alive – the OOM killer
Summary
Questions
Further reading
10
The CPU Scheduler – Part 1
The CPU Scheduler – Part 1
Technical requirements
Learning about the CPU scheduling internals – part 1 – essential background
Visualizing the flow
Learning about the CPU scheduling internals – part 2
Querying a given thread’s scheduling policy and priority
Learning about the CPU scheduling internals – part 3
Summary
Questions
Further reading
11
The CPU Scheduler – Part 2
The CPU Scheduler – Part 2
Technical requirements
Understanding, querying, and setting the CPU affinity mask
Querying and setting a thread’s scheduling policy and priority
An introduction to cgroups
Running Linux as an RTOS – an introduction
Miscellaneous scheduling related topics
Summary
Questions
Further reading
12
Kernel Synchronization – Part 1
Kernel Synchronization – Part 1
Technical requirements
Critical sections, exclusive execution, and atomicity
Concurrency concerns within the Linux kernel
Mutex or spinlock? Which to use when
Using the mutex lock
Using the spinlock
Locking and interrupts
Locking – common mistakes and guidelines
Solutions
Summary
Questions
Further reading
13
Kernel Synchronization – Part 2
Kernel Synchronization – Part 2
Technical requirements
Using the atomic_t and refcount_t interfaces
Using the RMW atomic operators
Using the reader-writer spinlock
Understanding CPU caching basics, cache effects, and false sharing
Lock-free programming with per-CPU and RCU
Lock debugging within the kernel
Introducing memory barriers
Summary
Questions
Further reading
14
Other Books You May Enjoy
Other Books You May Enjoy
15
Index
Index

Building the 6.x Linux Kernel from Source – Part 1

Building the Linux kernel from source code is an interesting way to begin your kernel development journey! Be assured, the journey’s a long and arduous one, but that’s the fun of it, right? The topic of kernel building itself is large enough to merit being divided into two chapters, this one and the next.

Recall what we have learned till now in the extended first chapter that has been published online (http://www.packtpub.com/sites/default/files/downloads/9781803232225_Online_Chapter.pdf): primarily, how to set the workspace for Linux kernel programming. You have also been introduced to user and kernel documentation sources and several useful projects that go hand-in-hand with kernel/driver development. By now, I assume you’ve completed the chapter that has been published online, and thus the setup of the workspace environment; if not, please do so before proceeding forward.

The primary purpose of this chapter and the next is to describe in detail how exactly you can build a modern Linux kernel from scratch using source code. In this chapter, you will first learn about the required basics: the kernel version nomenclature, development workflow, and the different types of source trees. Then, we’ll get hands-on: you’ll learn how to download a stable vanilla Linux kernel source tree onto a guest Linux Virtual Machine (VM). By “vanilla kernel,” we mean the plain and regular default kernel source code released by the Linux kernel community on its repository, https://www.kernel.org. After that, you will learn a little bit about the layout of the kernel source code – getting, in effect, a 10,000-foot view of the kernel code base. The actual kernel build recipe then follows.

Before proceeding, a key piece of information: any modern Linux system, be it a supercomputer or a tiny, embedded device, has three required components:

  • A bootloader
  • An Operating System (OS) kernel
  • A root filesystem

It additionally has two optional components:

  • If the processor family is ARM or PPC (32- or 64-bit), a Device Tree Blob (DTB) image file
  • An initramfs (or initrd) image file

In these two chapters, we concern ourselves only with the building of the OS (Linux) kernel from source code. We do not delve into the root filesystem details. In the next chapter, we will learn how to minimally configure the x86-specific GNU GRUB bootloader.

The complete kernel build process – for x86[_64] at least – requires a total of six or seven steps. Besides the required preliminaries, we cover the first three here and the remaining in the next chapter.

In this chapter, we will cover the following topics:

  • Preliminaries for the kernel build
  • Steps to build the kernel from source
  • Step 1 – Obtaining a Linux kernel source tree
  • Step 2 – Extracting the kernel source tree
  • Step 3 – Configuring the Linux kernel
  • Customizing the kernel menu, Kconfig, and adding our own menu item

You may wonder: what about building the Linux kernel for another CPU architecture (like ARM 32 or 64 bit)? We do precisely this as well in the following chapter!

Previous Section
End of Section 1
Next Section