Linux Kernel Programming Part 2 - Char Device Drivers and Kernel Synchronization

By : Kaiwan N. Billimoria

Linux Kernel Programming Part 2 - Char Device Drivers and Kernel Synchronization

By: Kaiwan N. Billimoria

Overview of this book

Linux Kernel Programming Part 2 - Char Device Drivers and Kernel Synchronization is an ideal companion guide to the Linux Kernel Programming book. This book provides a comprehensive introduction for those new to Linux device driver development and will have you up and running with writing misc class character device driver code (on the 5.4 LTS Linux kernel) in next to no time. You'll begin by learning how to write a simple and complete misc class character driver before interfacing your driver with user-mode processes via procfs, sysfs, debugfs, netlink sockets, and ioctl. You'll then find out how to work with hardware I/O memory. The book covers working with hardware interrupts in depth and helps you understand interrupt request (IRQ) allocation, threaded IRQ handlers, tasklets, and softirqs. You'll also explore the practical usage of useful kernel mechanisms, setting up delays, timers, kernel threads, and workqueues. Finally, you'll discover how to deal with the complexity of kernel synchronization with locking technologies (mutexes, spinlocks, and atomic/refcount operators), including more advanced topics such as cache effects, a primer on lock-free techniques, deadlock avoidance (with lockdep), and kernel lock debugging techniques. By the end of this Linux kernel book, you'll have learned the fundamentals of writing Linux character device driver code for real-world projects and products.

Preface

Who this book is for

What this book covers

To get the most out of this book

Download the example code files

Download the color images

Conventions used

Get in touch

Reviews

Section 1: Character Device Driver Basics

Free Chapter

Writing a Simple misc Character Device Driver

Technical requirements

Getting started with writing a simple misc character device driver

Understanding the device basics

A quick note on the Linux Device Model

Writing the misc driver code – part 1

Understanding the connection between the process, the driver, and the kernel

Handling unsupported methods

Writing the misc driver code – part 2

Writing the misc driver code – part 3

Testing our simple misc driver

Copying data from kernel to user space and vice versa

Leveraging kernel APIs to perform the data transfer

A misc driver with a secret

Writing the 'secret' misc device driver's code

Our secret driver – the init code

Our secret driver – the read method

Our secret driver – the write method

Our secret driver – cleanup

Our secret driver – the user space test app

Issues and security concerns

Hacking the secret driver

Bad driver – buggy read()

Bad driver – buggy write() – a privesc!

User space test app modifications

Device driver modifications

Let's get root now

Summary

Questions

Further reading

User-Kernel Communication Pathways

Technical requirements

Approaches to communicating/interfacing a kernel driver with a user space C app

Interfacing via the proc filesystem (procfs)

Understanding the proc filesystem

Directories under /proc

The purpose behind the proc filesystem

procfs is off-bounds to driver authors

Using procfs to interface with the user space

Basic procfs APIs

The four procfs files we will create

Trying out the dynamic debug_level procfs control

Dynamically controlling debug_level via procfs

A few misc procfs APIs

Interfacing via the sys filesystem (sysfs)

Creating a sysfs (pseudo) file in code

Creating a simple platform device

Platform devices

Tying it all together – setting up the device attributes and creating the sysfs file

The code for implementing our sysfs file and its callbacks

The "one value per sysfs file" rule

Interfacing via the debug filesystem (debugfs)

Checking for the presence of debugfs

Looking up the debugfs API documentation

An interfacing example with debugfs

Creating and using the first debugfs file

Creating and using the second debugfs file

Helper debugfs APIs for working on numeric globals

Removing the debugfs pseudo file(s)

Seeing a kernel bug – an Oops!

Debugfs – actual users

Interfacing via netlink sockets

Advantages using sockets

Understanding what a netlink socket is

Writing the user space netlink socket application

Writing the kernel-space netlink socket code as a kernel module

Trying out our netlink interfacing project

Interfacing via the ioctl system call

Using ioctl in the user and kernel space

User space – using the ioctl system call

Kernel space – using the ioctl system call

ioctl as a debug interface

Comparing the interfacing methods – a table

Summary

Questions

Further reading

Working with Hardware I/O Memory

Technical requirements

Accessing hardware I/O memory from the kernel

Understanding the issue with direct access

The solution – mapping via I/O memory or I/O port

Asking the kernel's permission

Understanding and using memory-mapped I/O

Using the ioremap*() APIs

The newer breed – the devm_* managed APIs

Obtaining the device resources

All in one with the devm_ioremap_resource() API

Looking up the new mapping via /proc/iomem

MMIO – performing the actual I/O

Performing 1- to 8-byte reads and writes on MMIO memory regions

Performing repeating I/O on MMIO memory regions

Setting and copying on MMIO memory regions

Understanding and using port-mapped I/O

PMIO – performing the actual I/O

A PIO example – the i8042

Looking up the port(s) via /proc/ioports

Port I/O – a few remaining points to note

Summary

Questions

Further reading

Handling Hardware Interrupts

Technical requirements

Hardware interrupts and how the kernel handles them

Allocating the hardware IRQ

Allocating your interrupt handler with request_irq()

Freeing the IRQ line

Setting interrupt flags

Understanding level- and edge-triggered interrupts – a brief note

Code view 1 – the IXGB network driver

Implementing the interrupt handler routine

Interrupt context guidelines – what to do and what not to do

Don't block – spotting possibly blocking code paths

Interrupt masking – the defaults and controlling it

Keep it fast

Writing the interrupt handler routine itself

Code view 2 – the i8042 driver's interrupt handler

Code view 3 – the IXGB network driver's interrupt handler

IRQ allocation – the modern way – the managed interrupt facility

Working with the threaded interrupts model

Employing the threaded interrupt model – the API

Employing the managed threaded interrupt model – the recommended way

Code view 4 – the STM32 F7 microcontroller's threaded interrupt handler

Internally implementing the threaded interrupt

Why use threaded interrupts?

Threaded interrupts – to really make it real time

Constraints when using a threaded handler

Working with either hardirq or threaded handlers

Enabling and disabling IRQs

The NMI

Viewing all allocated interrupt (IRQ) lines

Understanding and using top and bottom halves

Specifying and using a tasklet

Initializing the tasklet

Running the tasklet

Understanding the kernel softirq mechanism

Available softirqs and what they are for

Understanding how the kernel runs softirqs

Running tasklets

Employing the ksoftirqd kernel threads

Softirqs and concurrency

Hardirqs, tasklets, and threaded handlers – what to use when

Fully figuring out the context

Viewing the context – examples

How Linux prioritizes activities

A few remaining FAQs answered

Load balancing interrupts and IRQ affinity

Does the kernel maintain separate IRQ stacks?

Measuring metrics and latency

Measuring interrupts with [e]BPF

Measuring time servicing individual hardirqs

Measuring time servicing individual softirqs

Using Ftrace to get a handle on system latencies

Finding the interrupts disabled worst-case time latency with Ftrace

Other tools

Summary

Questions

Further reading

Working with Kernel Timers, Threads, and Workqueues

Technical requirements

Delaying for a given time in the kernel

Understanding how to use the *delay() atomic APIs

Understanding how to use the *sleep() blocking APIs

Taking timestamps within kernel code

Let's try it – how long do delays and sleeps really take?

The "sed" drivers – to demo kernel timers, kthreads, and workqueues

Setting up and using kernel timers

Using kernel timers

Our simple kernel timer module – code view 1

Our simple kernel timer module – code view 2

Our simple kernel timer module – running it

sed1 – implementing timeouts with our demo sed1 driver

Deliberately missing the bus

Creating and working with kernel threads

A simple demo – creating a kernel thread

Running the kthread_simple kernel thread demo

The sed2 driver – design and implementation

sed2 – the design

sed2 driver – code implementation

sed2 – trying it out

Querying and setting the scheduling policy/priority of a kernel thread

Using kernel workqueues

The bare minimum workqueue internals

Using the kernel-global workqueue

Initializing the kernel-global workqueue for your task – INIT_WORK()

Having your work task execute – schedule_work()

Variations of scheduling your work task

Cleaning up – canceling or flushing your work task

A quick summary of the workflow

Our simple work queue kernel module – code view

Our simple work queue kernel module – running it

The sed3 mini project – a very brief look

Summary

Questions

Further reading

Section 2: Delving Deeper

Kernel Synchronization - Part 1

Critical sections, exclusive execution, and atomicity

What is a critical section?

A classic case – the global i ++

Concepts – the lock

A summary of key points

Concurrency concerns within the Linux kernel

Multicore SMP systems and data races

Preemptible kernels, blocking I/O, and data races

Hardware interrupts and data races

Locking guidelines and deadlocks

Mutex or spinlock? Which to use when

Determining which lock to use – in theory

Determining which lock to use – in practice

Using the mutex lock

Initializing the mutex lock

Correctly using the mutex lock

Mutex lock and unlock APIs and their usage

Mutex lock – via [un]interruptible sleep?

Mutex locking – an example driver

The mutex lock – a few remaining points

Mutex lock API variants

The mutex trylock variant

The mutex interruptible and killable variants

The mutex io variant

The semaphore and the mutex

Priority inversion and the RT-mutex

Internal design

Using the spinlock

Spinlock – simple usage

Spinlock – an example driver

Test – sleep in an atomic context

Testing on a 5.4 debug kernel

Testing on a 5.4 non-debug distro kernel

Locking and interrupts

Using spinlocks – a quick summary

Summary

Questions

Further reading

Kernel Synchronization - Part 2

Using the atomic_t and refcount_t interfaces

The newer refcount_t versus older atomic_t interfaces

The simpler atomic_t and refcount_t interfaces

Examples of using refcount_t within the kernel code base

64-bit atomic integer operators

Using the RMW atomic operators

RMW atomic operations – operating on device registers

Using the RMW bitwise operators

Using bitwise atomic operators – an example

Efficiently searching a bitmask

Using the reader-writer spinlock

Reader-writer spinlock interfaces

A word of caution

The reader-writer semaphore

Cache effects and false sharing

Lock-free programming with per-CPU variables

Per-CPU variables

Working with per-CPU

Allocating, initialization, and freeing per-CPU variables

Performing I/O (reads and writes) on per-CPU variables

Per-CPU – an example kernel module

Per-CPU usage within the kernel

Lock debugging within the kernel

Configuring a debug kernel for lock debugging

The lock validator lockdep – catching locking issues early

Examples – catching deadlock bugs with lockdep

Example 1 – catching a self deadlock bug with lockdep

Fixing it

Example 2 – catching an AB-BA deadlock with lockdep

lockdep – annotations and issues

Memory barriers – an introduction

An example of using memory barriers in a device driver

Summary

Questions

Further reading

Other Books You May Enjoy

Leave a review - let other readers know what you think

Customer Reviews

5 star

4 star

3 star

2 star

1 star

Measuring time servicing individual softirqs

Similar to what we did previously with hardirqs, we will now employ the softirqs[-bpfcc] tool. It displays the total time spent servicing softirqs (software interrupts). Again, you will require root access to run these [e]BPF tools.

First, let's place our system (native x86_64 running Ubuntu) under some stress (here, it's performing network downloads, network uploads, and disk activity). The following screenshot shows us running the softirqs-bpfcc tool, which provides information about the total time spent servicing softirqs every 1 second (first parameter) forever (no second parameter):

Figure 4.16 – softirqs-bpfcc displaying the time that was spent servicing softirqs every 1 second (under some I/O stress)

Notice how the tasklet softirq also comes into play.

Let's look at another example of using the same tool to generate a histogram of soft IRQ time distribution...

Linux Kernel Programming Part 2 - Char Device Drivers and Kernel Synchronization

By : Kaiwan N. Billimoria

Linux Kernel Programming Part 2 - Char Device Drivers and Kernel Synchronization

By: Kaiwan N. Billimoria

Overview of this book

Related Content you might be interested in

Current Title:

Linux Kernel Programming Part 2 - Char Device Drivers and Kernel Synchronization

Linux Kernel Programming

Linux Kernel Debugging

Mastering Linux Kernel Development

Measuring time servicing individual softirqs