Book Image

Learn LLVM 17 - Second Edition

By : Kai Nacke, Amy Kwan
Book Image

Learn LLVM 17 - Second Edition

By: Kai Nacke, Amy Kwan

Overview of this book

LLVM was built to bridge the gap between the theoretical knowledge found in compiler textbooks and the practical demands of compiler development. With a modular codebase and advanced tools, LLVM empowers developers to build compilers with ease. This book serves as a practical introduction to LLVM, guiding you progressively through complex scenarios and ensuring that you navigate the challenges of building and working with compilers like a pro. The book starts by showing you how to configure, build, and install LLVM libraries, tools, and external projects. You’ll then be introduced to LLVM's design, unraveling its applications in each compiler stage: frontend, optimizer, and backend. Using a real programming language subset, you'll build a frontend, generate LLVM IR, optimize it through the pipeline, and generate machine code. Advanced chapters extend your expertise, covering topics such as extending LLVM with a new pass, using LLVM tools for debugging, and enhancing the quality of your code. You'll also focus on just-in-time compilation issues and the current state of JIT-compilation support with LLVM. Finally, you’ll develop a new backend for LLVM, gaining insights into target description and how instruction selection works. By the end of this book, you'll have hands-on experience with the LLVM compiler development framework through real-world examples and source code snippets.
Table of Contents (20 chapters)
1
Part 1: The Basics of Compiler Construction with LLVM
4
Part 2: From Source to Machine Code Generation
10
Part 3: Taking LLVM to the Next Level
14
Part 4: Roll Your Own Backend

Getting the prerequisites ready

To work with LLVM, your development system should run a common operating system such as Linux, FreeBSD, macOS, or Windows. You can build LLVM and clang in different modes. A build with debug symbols enabled can take up to 30 GB of space. The required disk space depends heavily on the chosen build options. For example, building only the LLVM core libraries in release mode, targeting only one platform, requires about 2 GB of free disk space, which is the bare minimum needed.

To reduce compile times, a fast CPU (such as a quad-core CPU with a 2.5 GHz clock speed) and a fast SSD are also helpful. It is even possible to build LLVM on a small device such as a Raspberry Pi – it only takes a lot of time. The examples within this book were developed on a laptop with an Intel quad-core CPU running at a 2.7 GHz clock speed, with 40 GB RAM and 2.5 TB SSD disk space. This system is well suited for the development task.

Your development system must have some prerequisite software installed. Let’s review the minimal required version of these software packages.

To check out the source from GitHub, you need Git (https://git-scm.com/). There is no requirement for a specific version. The GitHub help pages recommend using at least version 1.17.10. Due to known security issues found in the past, it is recommended to use the latest available version, which is 2.39.1 at the time of writing.

The LLVM project uses CMake (https://cmake.org/) as the build file generator. At least the 3.20.0 version is required. CMake can generate build files for various build systems. In this book, Ninja (https://ninja-build.org/) is used because it is fast and available on all platforms. The latest version, 1.11.1, is recommended.

Obviously, you also need a C/C++ compiler. The LLVM projects are written in modern C++, based on the C++17 standard. A conforming compiler and standard library are required. The following compilers are known to work with LLVM 17:

  • gcc 7.1.0 or later
  • clang 5.0 or later
  • Apple clang 10.0 or later
  • Visual Studio 2019 16.7 or later

Tip

Please be aware that with further development of the LLVM project, the requirements for the compiler are most likely to change. In general, you should use the latest compiler version available for your system.

Python (https://python.org/) is used during the generation of the build files and for running the test suite. It should be at least the 3.8 version.

Although not covered in this book, there can be reasons that you need to use Make instead of Ninja. In this case, you need to use GNU Make (https://www.gnu.org/software/make/) version 3.79 or later. The usage of both build tools is very similar. It is sufficient to replace ninja in each command with make for the scenarios described below.

LLVM also depends on the zlib library (https://www.zlib.net/). You should have at least version 1.2.3.4 installed. As usual, we recommend using the latest version, 1.2.13.

To install the prerequisite software, the easiest way is to use the package manager from your operating system. In the following sections, the commands required to install the software are shown for the most popular operating systems.

Ubuntu

Ubuntu 22.04 uses the apt package manager. Most of the basic utilities are already installed; only the development tools are missing. To install all packages at once, you type the following:

$ sudo apt -y install gcc g++ git cmake ninja-build zlib1g-dev

Fedora and RedHat

The package manager of Fedora 37 and RedHat Enterprise Linux 9 is called dnf. Like Ubuntu, most of the basic utilities are already installed. To install all packages at once, you type the following:

$ sudo dnf –y install gcc gcc-c++ git cmake ninja-build \
  zlib-devel

FreeBSD

On FreeBSD 13 or later, you have to use the pkg package manager. FreeBSD differs from Linux-based systems in that the clang compiler is already installed. To install all other packages at once, you type the following:

$ sudo pkg install –y git cmake ninja zlib-ng

OS X

For development on OS X, it is best to install Xcode from the Apple store. While the Xcode IDE is not used in this book, it comes with the required C/C++ compilers and supporting utilities. For installation of the other tools, the package manager Homebrew (https://brew.sh/) can be used. To install all packages at once, you type the following:

$ brew install git cmake ninja zlib

Windows

Like OS X, Windows does not come with a package manager. For the C/C++ compiler, you need to download Visual Studio Community 2022 (https://visualstudio.microsoft.com/vs/community/), which is free for personal use. Please make sure that you install the workload named Desktop Development with C++. You can use the package manager Scoop (https://scoop.sh/) to install the other packages. After installing Scoop as described on the website, you open x64 Native Tools Command Prompt for VS 2022 from your Windows menu. To install the required packages, you type the following:

$ scoop install git cmake ninja python gzip bzip2 coreutils
$ scoop bucket add extras
$ scoop install zlib

Please watch the output from Scoop closely. For the Python and zlib packages, it advises adding some registry keys. These entries are needed so that other software can find these packages. To add the registry keys, you’d best copy and paste the output from Scoop, which looks like the following:

$ %HOMEPATH%\scoop\apps\python\current\install-pep-514.reg
$ %HOMEPATH%\scoop\apps\zlib\current\register.reg

After each command, a message window from the registry editor will pop up asking whether you really want to import those registry keys. You need to click on Yes to finish the import. Now all prerequisites are installed.

For all examples in this book, you must use the x64 Native Tools Command Prompt for VS 2022. Using this command prompt, the compiler is automatically added to the search path.

Tip

The LLVM code base is very large. To comfortably navigate the source, we recommend using an IDE that allows you to jump to the definition of classes and search through the source. We find Visual Studio Code (https://code.visualstudio.com/download), which is an extensible cross-platform IDE, very comfortable to use. However, this is no requirement for following the examples in this book.