Book Image

Linux Kernel Programming Part 2 - Char Device Drivers and Kernel Synchronization

By : Kaiwan N. Billimoria
Book Image

Linux Kernel Programming Part 2 - Char Device Drivers and Kernel Synchronization

By: Kaiwan N. Billimoria

Overview of this book

Linux Kernel Programming Part 2 - Char Device Drivers and Kernel Synchronization is an ideal companion guide to the Linux Kernel Programming book. This book provides a comprehensive introduction for those new to Linux device driver development and will have you up and running with writing misc class character device driver code (on the 5.4 LTS Linux kernel) in next to no time. You'll begin by learning how to write a simple and complete misc class character driver before interfacing your driver with user-mode processes via procfs, sysfs, debugfs, netlink sockets, and ioctl. You'll then find out how to work with hardware I/O memory. The book covers working with hardware interrupts in depth and helps you understand interrupt request (IRQ) allocation, threaded IRQ handlers, tasklets, and softirqs. You'll also explore the practical usage of useful kernel mechanisms, setting up delays, timers, kernel threads, and workqueues. Finally, you'll discover how to deal with the complexity of kernel synchronization with locking technologies (mutexes, spinlocks, and atomic/refcount operators), including more advanced topics such as cache effects, a primer on lock-free techniques, deadlock avoidance (with lockdep), and kernel lock debugging techniques. By the end of this Linux kernel book, you'll have learned the fundamentals of writing Linux character device driver code for real-world projects and products.

Related Content you might be interested in

Current Title:

Small book image
Linux Kernel Programming Part 2 - Char Device Drivers and Kernel Synchronization
Kaiwan N. Billimoria
Mar, 2021 | 452 pages
Small book image
Linux Kernel Programming
Kaiwan N Billimoria
Feb, 2024 | 826 pages
Small book image
Linux Kernel Debugging
Kaiwan N Billimoria
Aug, 2022 | 638 pages
Small book image
Mastering Linux Kernel Development
CH Raghav Maruthi
Oct, 2017 | 354 pages
Table of Contents (11 chapters)
Preface
Preface
Who this book is for
What this book covers
To get the most out of this book
Download the example code files
Download the color images
Conventions used
Get in touch
Reviews
1
Section 1: Character Device Driver Basics
Section 1: Character Device Driver Basics
Free Chapter
2
Writing a Simple misc Character Device Driver
Writing a Simple misc Character Device Driver
Technical requirements
Getting started with writing a simple misc character device driver
Understanding the device basics
A quick note on the Linux Device Model
Writing the misc driver code – part 1
Understanding the connection between the process, the driver, and the kernel
Handling unsupported methods
Writing the misc driver code – part 2
Writing the misc driver code – part 3
Testing our simple misc driver
Copying data from kernel to user space and vice versa
Leveraging kernel APIs to perform the data transfer
A misc driver with a secret
Writing the 'secret' misc device driver's code
Our secret driver – the init code
Our secret driver – the read method
Our secret driver – the write method
Our secret driver – cleanup
Our secret driver – the user space test app
Issues and security concerns
Hacking the secret driver
Bad driver – buggy read()
Bad driver – buggy write() – a privesc!
User space test app modifications
Device driver modifications
Let's get root now
Summary
Questions
Further reading
3
User-Kernel Communication Pathways
User-Kernel Communication Pathways
Technical requirements
Approaches to communicating/interfacing a kernel driver with a user space C app
Interfacing via the proc filesystem (procfs)
Understanding the proc filesystem
Directories under /proc
The purpose behind the proc filesystem
procfs is off-bounds to driver authors
Using procfs to interface with the user space
Basic procfs APIs
The four procfs files we will create
Trying out the dynamic debug_level procfs control
Dynamically controlling debug_level via procfs
A few misc procfs APIs
Interfacing via the sys filesystem (sysfs)
Creating a sysfs (pseudo) file in code
Creating a simple platform device
Platform devices
Tying it all together – setting up the device attributes and creating the sysfs file
The code for implementing our sysfs file and its callbacks
The "one value per sysfs file" rule
Interfacing via the debug filesystem (debugfs)
Checking for the presence of debugfs
Looking up the debugfs API documentation
An interfacing example with debugfs
Creating and using the first debugfs file
Creating and using the second debugfs file
Helper debugfs APIs for working on numeric globals
Removing the debugfs pseudo file(s)
Seeing a kernel bug – an Oops!
Debugfs – actual users
Interfacing via netlink sockets
Advantages using sockets
Understanding what a netlink socket is
Writing the user space netlink socket application
Writing the kernel-space netlink socket code as a kernel module
Trying out our netlink interfacing project
Interfacing via the ioctl system call
Using ioctl in the user and kernel space
User space – using the ioctl system call
Kernel space – using the ioctl system call
ioctl as a debug interface
Comparing the interfacing methods – a table
Summary
Questions
Further reading
4
Working with Hardware I/O Memory
Working with Hardware I/O Memory
Technical requirements
Accessing hardware I/O memory from the kernel
Understanding the issue with direct access
The solution – mapping via I/O memory or I/O port
Asking the kernel's permission
Understanding and using memory-mapped I/O
Using the ioremap*() APIs
The newer breed – the devm_* managed APIs
Obtaining the device resources
All in one with the devm_ioremap_resource() API
Looking up the new mapping via /proc/iomem
MMIO – performing the actual I/O
Performing 1- to 8-byte reads and writes on MMIO memory regions
Performing repeating I/O on MMIO memory regions
Setting and copying on MMIO memory regions
Understanding and using port-mapped I/O
PMIO – performing the actual I/O
A PIO example – the i8042
Looking up the port(s) via /proc/ioports
Port I/O – a few remaining points to note
Summary
Questions
Further reading
5
Handling Hardware Interrupts
Handling Hardware Interrupts
Technical requirements
Hardware interrupts and how the kernel handles them
Allocating the hardware IRQ
Allocating your interrupt handler with request_irq()
Freeing the IRQ line
Setting interrupt flags
Understanding level- and edge-triggered interrupts – a brief note
Code view 1 – the IXGB network driver
Implementing the interrupt handler routine
Interrupt context guidelines – what to do and what not to do
Don't block – spotting possibly blocking code paths
Interrupt masking – the defaults and controlling it
Keep it fast
Writing the interrupt handler routine itself
Code view 2 – the i8042 driver's interrupt handler
Code view 3 – the IXGB network driver's interrupt handler
IRQ allocation – the modern way – the managed interrupt facility
Working with the threaded interrupts model
Employing the threaded interrupt model – the API
Employing the managed threaded interrupt model – the recommended way
Code view 4 – the STM32 F7 microcontroller's threaded interrupt handler
Internally implementing the threaded interrupt
Why use threaded interrupts?
Threaded interrupts – to really make it real time
Constraints when using a threaded handler
Working with either hardirq or threaded handlers
Enabling and disabling IRQs
The NMI
Viewing all allocated interrupt (IRQ) lines
Understanding and using top and bottom halves
Specifying and using a tasklet
Initializing the tasklet
Running the tasklet
Understanding the kernel softirq mechanism
Available softirqs and what they are for
Understanding how the kernel runs softirqs
Running tasklets
Employing the ksoftirqd kernel threads
Softirqs and concurrency
Hardirqs, tasklets, and threaded handlers – what to use when
Fully figuring out the context
Viewing the context – examples
How Linux prioritizes activities
A few remaining FAQs answered
Load balancing interrupts and IRQ affinity
Does the kernel maintain separate IRQ stacks?
Measuring metrics and latency
Measuring interrupts with [e]BPF
Measuring time servicing individual hardirqs
Measuring time servicing individual softirqs
Using Ftrace to get a handle on system latencies
Finding the interrupts disabled worst-case time latency with Ftrace
Other tools
Summary
Questions
Further reading
6
Working with Kernel Timers, Threads, and Workqueues
Working with Kernel Timers, Threads, and Workqueues
Technical requirements
Delaying for a given time in the kernel
Understanding how to use the *delay() atomic APIs
Understanding how to use the *sleep() blocking APIs
Taking timestamps within kernel code
Let's try it – how long do delays and sleeps really take?
The "sed" drivers – to demo kernel timers, kthreads, and workqueues
Setting up and using kernel timers
Using kernel timers
Our simple kernel timer module – code view 1
Our simple kernel timer module – code view 2
Our simple kernel timer module – running it
sed1 – implementing timeouts with our demo sed1 driver
Deliberately missing the bus
Creating and working with kernel threads
A simple demo – creating a kernel thread
Running the kthread_simple kernel thread demo
The sed2 driver – design and implementation
sed2 – the design
sed2 driver – code implementation
sed2 – trying it out
Querying and setting the scheduling policy/priority of a kernel thread
Using kernel workqueues
The bare minimum workqueue internals
Using the kernel-global workqueue
Initializing the kernel-global workqueue for your task – INIT_WORK()
Having your work task execute – schedule_work()
Variations of scheduling your work task
Cleaning up – canceling or flushing your work task
A quick summary of the workflow
Our simple work queue kernel module – code view
Our simple work queue kernel module – running it
The sed3 mini project – a very brief look
Summary
Questions
Further reading
7
Section 2: Delving Deeper
Section 2: Delving Deeper
8
Kernel Synchronization - Part 1
Kernel Synchronization - Part 1
Critical sections, exclusive execution, and atomicity
What is a critical section?
A classic case – the global i ++
Concepts – the lock
A summary of key points
Concurrency concerns within the Linux kernel
Multicore SMP systems and data races
Preemptible kernels, blocking I/O, and data races
Hardware interrupts and data races
Locking guidelines and deadlocks
Mutex or spinlock? Which to use when
Determining which lock to use – in theory
Determining which lock to use – in practice
Using the mutex lock
Initializing the mutex lock
Correctly using the mutex lock
Mutex lock and unlock APIs and their usage
Mutex lock – via [un]interruptible sleep?
Mutex locking – an example driver
The mutex lock – a few remaining points
Mutex lock API variants
The mutex trylock variant
The mutex interruptible and killable variants
The mutex io variant
The semaphore and the mutex
Priority inversion and the RT-mutex
Internal design
Using the spinlock
Spinlock – simple usage
Spinlock – an example driver
Test – sleep in an atomic context
Testing on a 5.4 debug kernel
Testing on a 5.4 non-debug distro kernel
Locking and interrupts
Using spinlocks – a quick summary
Summary
Questions
Further reading
9
Kernel Synchronization - Part 2
Kernel Synchronization - Part 2
Using the atomic_t and refcount_t interfaces
The newer refcount_t versus older atomic_t interfaces
The simpler atomic_t and refcount_t interfaces
Examples of using refcount_t within the kernel code base
64-bit atomic integer operators
Using the RMW atomic operators
RMW atomic operations – operating on device registers
Using the RMW bitwise operators
Using bitwise atomic operators – an example
Efficiently searching a bitmask
Using the reader-writer spinlock
Reader-writer spinlock interfaces
A word of caution
The reader-writer semaphore
Cache effects and false sharing
Lock-free programming with per-CPU variables
Per-CPU variables
Working with per-CPU
Allocating, initialization, and freeing per-CPU variables
Performing I/O (reads and writes) on per-CPU variables
Per-CPU – an example kernel module
Per-CPU usage within the kernel
Lock debugging within the kernel
Configuring a debug kernel for lock debugging
The lock validator lockdep – catching locking issues early
Examples – catching deadlock bugs with lockdep
Example 1 – catching a self deadlock bug with lockdep
Fixing it
Example 2 – catching an AB-BA deadlock with lockdep
lockdep – annotations and issues
lockdep annotations
lockdep issues
Lock statistics
Viewing lock stats
Memory barriers – an introduction
An example of using memory barriers in a device driver
Summary
Questions
Further reading
10
Other Books You May Enjoy
Other Books You May Enjoy
Leave a review - let other readers know what you think

Let's get root now

Of course, the proof of the pudding is in the eating, yes? So, let's give our hack a spin; here, we assume that you've first unloaded any previous version of our 'misc' drivers, and built and loaded the bad_miscdrv kernel module into memory:

Figure 1.11 – Screenshot showing our bad_miscdrv misc driver performing a "bad" write, resulting in root – a privesc!

Check it out; we indeed got root! Our rdwr_test_hackit app, detecting that we do have root (via a simple getuid(2)system call), then does the logical thing: it execs a root shell (via an execl(3) API), and voilà, we land up in a root shell. We show the kernel log:

$ dmesg 
[ 63.847549] bad_miscdrv:bad_miscdrv_init(): LLKD 'bad' misc driver (major # 10) registered, minor# = 56
[ 63.848452] misc bad_miscdrv: A sample print via the dev_dbg(): (bad) driver initialized
[ 84.186882] bad_miscdrv:open_miscdrv_rdwr(): 000) rdwr_test_hacki :2765 | ...0 /* open_miscdrv_rdwr() */
[ 84.190521] misc bad_miscdrv: opening "bad_miscdrv" now; wrt open file: f_flags = 0x8001
[ 84.191557] bad_miscdrv:write_miscdrv_rdwr(): 000) rdwr_test_hacki :2765 | ...0 /* write_miscdrv_rdwr() */
[ 84.192358] misc bad_miscdrv: rdwr_test_hacki wants to write 4 bytes to (original) ubuf = 0x55648b8f36b0
[ 84.192971] misc bad_miscdrv: [current->cred=ffff9f67765c3b40]
[ 84.193392] misc bad_miscdrv: dest addr = ffff9f67765c3b44 count=4
[ 84.193803] misc bad_miscdrv: 4 bytes written, returning... (stats: tx=0, rx=4)
[ 89.002675] bad_miscdrv:close_miscdrv_rdwr(): 000) [sh]:2765 | ...0 /* close_miscdrv_rdwr() */
[ 89.005992] misc bad_miscdrv: filename: "bad_miscdrv"
$

You can see how it's worked: the original user-mode buffer ubuf kernel virtual address is 0x55648b8f36b0. In the hack, we modify it to the new destination address (kernel virtual address), 0xffff9f67765c3b44, which is (in this case) the kernel virtual address of the UID member of struct cred (within the process's task structure). Not only that, but our driver also modifies the number of bytes to write (count) to 4 (bytes), as we're updating a 32-bit quantity.

Do note: these hacks are just that – hacks. They could certainly cause your system to become unstable (when run on our "debug" kernel, KASAN, in fact, detected a null pointer dereference!).

These demos prove nothing but the fact that you as a kernel and/or driver author must be alert to programming issues, security, and more at all times. With this, we complete this section and indeed the chapter.

Previous Section
End of Section 27
Next Section