Book Image

Linux Kernel Programming - Second Edition

By : Kaiwan N. Billimoria
Book Image

Linux Kernel Programming - Second Edition

By: Kaiwan N. Billimoria

Overview of this book

The 2nd Edition of Linux Kernel Programming is an updated, comprehensive guide for new programmers to the Linux kernel. This book uses the recent 6.1 Long-Term Support (LTS) Linux kernel series, which will be maintained until Dec 2026, and also delves into its many new features. Further, the Civil Infrastructure Project has pledged to maintain and support this 6.1 Super LTS (SLTS) kernel right until August 2033, keeping this book valid for years to come! You’ll begin this exciting journey by learning how to build the kernel from source. In a step by step manner, you will then learn how to write your first kernel module by leveraging the kernel’s powerful Loadable Kernel Module (LKM) framework. With this foundation, you will delve into key kernel internals topics including Linux kernel architecture, memory management, and CPU (task) scheduling. You’ll finish with understanding the deep issues of concurrency, and gain insight into how they can be addressed with various synchronization/locking technologies (e.g., mutexes, spinlocks, atomic/refcount operators, rw-spinlocks and even lock-free technologies such as per-CPU and RCU). By the end of this book, you’ll have a much better understanding of the fundamentals of writing the Linux kernel and kernel module code that can straight away be used in real-world projects and products.
Table of Contents (16 chapters)
14
Other Books You May Enjoy
15
Index

Customizing the kernel menu, Kconfig, and adding our own menu item

So, let’s say you have developed a device driver, an experimental new modular scheduling class, a custom debugfs (debug filesystem) callback, or some other cool kernel feature. You will one day. How will you let others on the team – or, for that matter, your customer or the community – know that this fantastic new kernel feature exists? You will typically set up a new kernel config (macro) and allow folks to select it as either a built-in or as a kernel module, and thus build and make use of it. As part of this, you’ll need to define the new kernel config and insert a new menu item at an appropriate place in the kernel configuration menu.

To do so, it’s useful to first understand a little more about the various Kconfig* files and where they reside. Let’s find out.

Understanding the Kconfig* files

The Kconfig* files contain metadata interpreted by the kernel’s config and build system – Kconfig/Kbuild – allowing it to build and (conditionally) display the menus you see when you run the menuconfig UI, accept selections, and so on.

For example, the Kconfig file at the root of the kernel source tree is used to fill in the initial screen of the menuconfig UI. Take a peek at it; it works by sourcing various other Kconfig files in different folders of the kernel source tree. There are many Kconfig* files within the kernel source tree (over 1,700 for 6.1.25)! Each of them typically defines a single menu, helping us realize how intricate the build system really is.

As a real example, let’s look up the Kconfig entry that defines the following items in the menu: Google Devices | Google Virtual NIC (gVNIC) support. Google Cloud employs a virtual Network Interface Card (NIC); it’s called the Google Virtual NIC. It’s likely that their Linux-based cloud servers will make use of it. Its location within the menuconfig UI is here:

-> Device Drivers

-> Network device support

-> Ethernet driver support

-> Google Devices

Here’s a screenshot showing these menu items:

Figure 2.19: Partial screenshot showing the Google Devices item in the make menuconfig UI (for x86/6.1.25)

How do we know which Kconfig file defines these menu items? The Help screen for a given config reveals it! So, while on the relevant menu item, select the < Help > button and press Enter; here, the Help screen says (among other things):

Defined at drivers/net/ethernet/google/Kconfig:5

That’s the Kconfig file describing this menu! Let’s look it up:

$ cat drivers/net/ethernet/google/Kconfig
#
# Google network device configuration
#
config NET_VENDOR_GOOGLE
    bool "Google Devices"
    default y
    help
      If you have a network (Ethernet) device belonging to this class, say Y.
[ … ]

This is a nice and simple Kconfig entry; notice the Kconfig language keywords: config, bool, default, and help. We’ve highlighted them in reverse colors. You can see that this device is enabled by default. We’ll cover the syntax shortly.

The following table summarizes the more important Kconfig* files and which submenu they serve in the Kbuild UI:

Menu

Kconfig file location for it

The main menu, the initial screen of the menuconfig UI

Kconfig

General setup + Enable loadable module support

init/Kconfig

Processor types and features + Bus options + Binary Emulations (this menu title tends to be arch-specific; here,its wrt the x86[_64]; in general, the Kconfig file is here: arch/<arch>/Kconfig)

arch/<arch>/Kconfig*

Power management

kernel/power/Kconfig*

Firmware drivers

drivers/firmware/Kconfig*

Virtualization

arch/<arch>/kvm/Kconfig*

General architecture-dependent options

arch/Kconfig*

Enable the block layer + IO Schedulers

block/Kconfig*

Executable file formats

fs/Kconfig.binfmt

Memory management options

mm/Kconfig*

Networking support

net/Kconfig, net/*/Kconfig*

Device drivers

drivers/Kconfig, drivers/*/Kconfig*

Filesystems

fs/Kconfig, fs/*/Kconfig*

Security options

security/Kconfig, security/*/Kconfig*

Cryptographic API

crypto/Kconfig, crypto/*/Kconfig*

Library routines

lib/Kconfig, lib/*/Kconfig*

Kernel hacking (implies Kernel debugging)

lib/Kconfig.debug, lib/Kconfig.*

Table 2.4: Kernel config (sub) menus and the corresponding Kconfig* file(s) defining them

Typically, a single Kconfig file drives a single menu, though there could be multiple. Now, let’s move on to actually adding a menu item.

Creating a new menu item within the General Setup menu

As a trivial example, let’s add our own Boolean dummy config option within the General Setup menu. We want the config name to be, shall we say, CONFIG_LKP_OPTION1. As can be seen from the preceding table, the relevant Kconfig file to edit is the init/Kconfig one as it’s the meta-file that defines the General Setup menu.

Let’s get to it (we assume you’re in the root of the kernel source tree):

  1. Optional: to be safe, always make a backup copy of the Kconfig file you’re editing:
    cp init/Kconfig init/Kconfig.orig
    
  2. Now, edit the init/Kconfig file:
    vi init/Kconfig
    

    Scroll down to an appropriate location within the file; here, we choose to insert our custom menu entry between the LOCALVERSION_AUTO and the BUILD_SALT ones. The following screenshot shows our new entry (the init/Kconfig file being edited with vim):

    Figure 2.20: Editing the 6.1.25:init/Kconfig and inserting our own menu entry (highlighted)

    FYI, I’ve provided the preceding experiment as a patch to the original 6.1.25 init/Kconfig file in our book’s GitHub source tree. Find the patch file here: ch2/Kconfig.patch.

    The new item starts with the config keyword followed by the FOO part of your new CONFIG_LKP_OPTION1 config variable. For now, just read the statements we have made in the Kconfig file regarding this entry. More details on the Kconfig language/syntax are in the A few details on the Kconfig language section that follows.

  1. Save the file and exit the editor.
  2. (Re)configure the kernel: run make menuconfig. Then navigate to our cool new menu item under General Setup | Test case for LKP 2e book/Ch 2: creating …. Turn the feature on. Notice how, in Figure 2.21, it’s highlighted and off by default, just as we specified via the default n line.
        make menuconfig
       [...]
    

    Here’s the relevant output:

    Figure 2.21: Kernel configuration via make menuconfig showing our new menu entry (before turning it on)

  1. Now turn it on by toggling it with the space bar, then save and exit the menu system.

    While there, try pressing the < Help > button. You should see the “help” text we provided within the init/Kconfig file.

  1. Check whether our feature has been selected:
    $ grep "LKP_OPTION1" .config
    CONFIG_LKP_OPTION1=y
    $ grep "LKP_OPTION1" include/generated/autoconf.h 
    $
    

    We find that indeed it has been set to on (y) within our .config file, but is not yet within the kernel’s internal auto-generated header file. This will happen when we build the kernel.

    Now let’s check it via the useful non-interactive config script method. We covered this in the Using the kernel’s config script to view/edit the kernel config section.

    $ scripts/config -s LKP_OPTION1
    y
    

    Ah, it’s on, as expected (the -s option is the same as --state). Below, we disable it via the -d option, query it (-s), and then re-enable it via the -e option, and again query it (just for learning’s sake!):

    $ scripts/config -d LKP_OPTION1 ; scripts/config -s LKP_OPTION1
    n
    $ scripts/config -e LKP_OPTION1 ; scripts/config -s LKP_OPTION1
    y
    
  1. Build the kernel. Worry not; the full details on building the kernel are found in the next chapter. You can skip this for now, or you could always cover Chapter 3, Building the 6.x Linux Kernel from Source – Part 2, and then come back to this point.
    make -j4
    

    Further, in recent kernels, after the build step, every kernel config option that’s enabled (either y or m) appears as an empty file within include/config; this happens with our new config as well, of course:

    $ ls -l include/config/LKP_*
    -rw-r--r-- 1 c2kp c2kp 0 Apr 29 11:56 include/config/LKP_OPTION1
    
  1. Once done, recheck the autoconf.h header for the presence of our new config option:
    $ grep LKP_OPTION1 include/generated/* 2>/dev/null 
    include/generated/autoconf.h:#define CONFIG_LKP_OPTION1 1
    include/generated/rustc_cfg:--cfg=CONFIG_LKP_OPTION1
    include/generated/rustc_cfg:--cfg=CONFIG_LKP_OPTION1="y"
    

It worked (from 6.0 even Rust knows about it!). Yes; however, when working on an actual project or product, in order to leverage this new kernel config of ours, we would typically require a further step, setting up our config entry within the Makefile relevant to the code that uses this config option.

Here’s a quick example of how this might look. Let’s imagine we wrote some kernel code in a C source file named lkp_options.c. Now, we need to ensure it gets compiled and built into the kernel image! How? Here’s one way: in the kernel’s top-level (or within its own) Makefile, the following line will ensure that it gets compiled into the kernel at build time; add it to the end of the relevant Makefile:

obj-${CONFIG_LKP_OPTION1}  +=  lkp_option1.o

Don’t stress about the fairly weird kernel Makefile syntax for now. The next few chapters will certainly shed some light on this. Also, we did cover this particular syntax in the How the Kconfig+Kbuild system works – a minimal take section.

Further, you should realize that the very same config can be used as a normal C macro within a piece of kernel code; for example, we could do things like this within our in-tree kernel (or module) C code:

#ifdef CONFIG_LKP_OPTION1
    do_our_thing();
#endif

Then again, it’s very much worth noting that the Linux kernel community has devised and strictly adheres to certain rigorous coding style guidelines. In this context, the guidelines state that conditional compilation should be avoided whenever possible; if it’s required to use a Kconfig symbol as a conditional, then please do it this way:

if (IS_ENABLED(CONFIG_LKP_OPTION1))
    do_our_thing();

The Linux kernel coding style guidelines can be found here: https://www.kernel.org/doc/html/latest/process/coding-style.html. I urge you to refer to them often, and, of course, to follow them!

A few details on the Kconfig language

Our usage of the Kconfig language so far (Figure 2.20) is just the tip of the proverbial iceberg. The fact is, the Kconfig system uses the Kconfig language (or syntax) to express and create menus using simple ASCII text directives. The language includes menu entries, attributes, dependencies, visibility constraints, help text, and so on.

The kernel documents the Kconfig language constructs and syntax here: https://www.kernel.org/doc/html/v6.1/kbuild/kconfig-language.html#kconfig-language. Do refer to this document for complete details.

A brief mention of the more common Kconfig constructs is given in the following table:

Construct

Meaning

config <FOO>

Specifies the menu entry name of the form CONFIG_FOO here; just use the FOO part after the config keyword.

Menu attributes

bool ["<description>"]

Specifies the config option as a Boolean; its value in .config will be either y (built into the kernel image) or will not exist (will show up as a commented-out entry).

tristate ["description>"]

Specifies the config option as tristate; its value in .config will be either y or m (built as a kernel module) or will not exist (will show up as a commented-out entry).

int ["<description>"]

Specifies the config option as taking an integer value.

range x-y

For an integer whose valid range is from x to y.

default <value>

Specifies the default value; use y, m, n, or other, as required.

prompt "<description>" [if <expr>]

An input prompt with a describing sentence (can be made conditional); a menu entry can have at most one prompt.

depends on "expr"

Defines a dependency for the menu item; can have several with the depends on FOO1 && FOO2 && (FOO3 || FOO4) type of syntax.

select <config> [if "expr"]

Defines a reverse dependency.

help "my awesome help text … bleh

bleh bleh "

Text to display when the < Help > button is selected.

Table 2.5: Kconfig, a few constructs

To help understand the syntax, a few examples from lib/Kconfig.debug (the file that describes the menu items for the Kernel Hacking submenu – it means kernel debugging, really – of the UI) follow (don’t forget, you can browse it online as well: https://elixir.bootlin.com/linux/v6.1.25/source/lib/Kconfig.debug):

  1. We will start with a simple and self-explanatory one (the CONFIG_DEBUG_INFO option):
    config DEBUG_INFO
            bool
            help
              A kernel debug info option other than "None" has been selected
              in the "Debug information" choice below, indicating that debug
              information will be generated for build targets.
    
  2. Next, let’s look at the CONFIG_FRAME_WARN option. Notice the range and the conditional default value syntax, as follows:
    config FRAME_WARN
        int "Warn for stack frames larger than"
        range 0 8192
        default 0 if KMSAN
            default 2048 if GCC_PLUGIN_LATENT_ENTROPY
            default 2048 if PARISC
            default 1536 if (!64BIT && XTENSA)
            default 1280 if KASAN && !64BIT
            default 1024 if !64BIT
            default 2048 if 64BIT
            help
              Tell the compiler to warn at build time for stack frames larger than this.
              Setting this too low will cause a lot of warnings.
              Setting it to 0 disables the warning.
    
  3. Next, the CONFIG_HAVE_DEBUG_STACKOVERFLOW option is a simple Boolean; it’s either on or off (the kernel either has the capability to detect kernel-space stack overflows or doesn’t). The CONFIG_DEBUG_STACKOVERFLOW option is also a Boolean. Notice how it depends on two other options, separated with a Boolean AND (&&) operator:
    config HAVE_DEBUG_STACKOVERFLOW
            bool
    config DEBUG_STACKOVERFLOW
            bool "Check for stack overflows"
            depends on DEBUG_KERNEL && HAVE_DEBUG_STACKOVERFLOW
            help
               Say Y here if you want to check for overflows of kernel, IRQ
               and exception stacks (if your architecture uses them). This 
               option will show detailed messages if free stack space drops
               below a certain limit. [...]
    

Another useful thing: while configuring the kernel (via the usual make menuconfig UI), clicking on < Help > not only shows some (usually useful) help text, but it also displays the current runtime values of various config options. The same can be seen by simply searching for a config option (via the slash key, /, as mentioned earlier). So, for example, type / and search for the kernel config named KASAN; this is what I see when doing so.

Figure 2.22: Partial screenshot showing the KASAN config option; you can see it’s off by default

If you’re unaware, KASAN is the Kernel Address SANitizer – it’s a brilliant compiler-based technology to help catch memory corruption defects; I cover it in depth in the book Linux Kernel Debugging.

Look carefully at the Depends on: line; it shows the dependencies as well as their current value. The important thing to note is that the menu item won’t even show in the UI unless the dependencies are fulfilled.

Alright! This completes our coverage of the Kconfig files, creating or editing a custom menu entry in the kernel config, a little Kconfig language syntax, and indeed this chapter.