Book Image

Learn LLVM 12

By : Kai Nacke
Book Image

Learn LLVM 12

By: Kai Nacke

Overview of this book

LLVM was built to bridge the gap between compiler textbooks and actual compiler development. It provides a modular codebase and advanced tools which help developers to build compilers easily. This book provides a practical introduction to LLVM, gradually helping you navigate through complex scenarios with ease when it comes to building and working with compilers. You’ll start by configuring, building, and installing LLVM libraries, tools, and external projects. Next, the book will introduce you to LLVM design and how it works in practice during each LLVM compiler stage: frontend, optimizer, and backend. Using a subset of a real programming language as an example, you will then learn how to develop a frontend and generate LLVM IR, hand it over to the optimization pipeline, and generate machine code from it. Later chapters will show you how to extend LLVM with a new pass and how instruction selection in LLVM works. You’ll also focus on Just-in-Time compilation issues and the current state of JIT-compilation support that LLVM provides, before finally going on to understand how to develop a new backend for LLVM. By the end of this LLVM book, you will have gained real-world experience in working with the LLVM compiler development framework with the help of hands-on examples and source code snippets.
Table of Contents (17 chapters)
1
Section 1 – The Basics of Compiler Construction with LLVM
5
Section 2 – From Source to Machine Code Generation
11
Section 3 –Taking LLVM to the Next Level

Customizing the build process

The CMake system uses a project description in the CMakeLists.txt file. The top-level file is in the llvm directory; that is, llvm/CMakeLists.txt. Other directories also contain CMakeLists.txt files, which are recursively included during the build-file generation.

Based on the information provided in the project description, CMake checks which compilers have been installed, detects libraries and symbols, and creates the build system files, such as build.ninja or Makefile (depending on the chosen generator). It is also possible to define reusable modules, such as a function to detect if LLVM is installed. These scripts are placed in the special cmake directory (llvm/cmake), which is searched automatically during the generation process.

The build process can be customized by defining CMake variables. The–D command-line option is used to set a variable to a value. These variables are used in CMake scripts. Variables defined by CMake itself are almost always prefixed with CMAKE_, and these variables can be used in all projects. Variables defined by LLVM are prefixed with LLVM_ but they can only be used if the project definition includes the use of LLVM.

Variables defined by CMake

Some variables are initialized with the values of environment variables. The most notable are CC and CXX, which define the C and C++ compilers to be used for building. CMake tries to locate a C and a C++ compiler automatically, using the current shell search path. It picks the first compiler that's found. If you have several compilers installed, such as gcc and Clang or different versions of Clang, then this might not be the compiler you want for building LLVM.

Suppose you like to use clang9 as a C compiler and clang++9 as a C++ compiler. Here, you can invoke CMake in a Unix shell in the following way:

$ CC=clang9 CXX=clang++9 cmake ../llvm

This sets the value of the environment variables for the invocation of cmake. If necessary, you can specify an absolute path for the compiler executables.

CC is the default value of the CMAKE_C_COMPILER CMake variable, while CXX is the default value of the CMAKE_CXX_COMPILER CMake variable. Instead of using the environment variables, you can set the CMake variables directly. This is equivalent to the preceding call:

$ cmake –DCMAKE_C_COMPILER=clang9\
  -DCMAKE_CXX_COMPILER=clang++9 ../llvm

Other useful variables defined by CMake are as follows:

  • CMAKE_INSTALL_PREFIX: A path prefix that is prepended to every path during installation. The default is /usr/local on Unix and C:\Program Files\<Project> on Windows. To install LLVM in the /opt/llvm directory, you must specify -DCMAKE_INSTALL_PREFIX=/opt/llvm. The binaries are copied to /opt/llvm/bin, the library files are copied to /opt/llvm/lib, and so on.
  • CMAKE_BUILD_TYPE: Different types of builds require different settings. For example, a debug build needs to specify options for generating debug symbols and are usually linking against debug versions of system libraries. In contrast, a release build uses optimization flags and links against production versions of libraries. This variable is only used for build systems that can only handle one build type, such as Ninja or Make. For IDE build systems, all variants are generated, and you must use the mechanism of the IDE to switch between build types. Some possible values are as follows:

    DEBUG: Build with debug symbols

    RELEASE: Build with optimization for speed

    RELWITHDEBINFO: Release build with debug symbols

    MINSIZEREL: Build with optimization for size

    The default build type is DEBUG. To generate build files for a release build, you must specify -DCMAKE_BUILD_TYPE=RELEASE.

  • CMAKE_C_FLAGS and CMAKE_CXX_FLAGS: These are extra flags that are used when we're compiling C and C++ source files. The initial values are taken from the CFLAGS and CXXFLAGS environment variables, which can be used as alternatives.
  • CMAKE_MODULE_PATH: Specifies additional directories that are searched for in CMake modules. The specified directories are searched before the default ones. The value is a semicolon-separated list of directories.
  • PYTHON_EXECUTABLE: If the Python interpreter is not found or if the wrong one is picked if you have installed multiple versions of it, you can set this variable to the path of the Python binary. This variable only takes effect if the Python module of CMake is included (which is the case for LLVM).

CMake provides built-in help for variables. The --help-variable var option prints help for the var variable. For instance, you can type the following to get help for CMAKE_BUILD_TYPE:

$ cmake --help-variable CMAKE_BUILD_TYPE

You can also list all the variables with the following command:

$ cmake --help-variablelist

This list is very long. You may want to pipe the output to more or a similar program.

Variables defined by LLVM

The variables defined by LLVM work in the same way as those defined by CMake, except that there is no built-in help. The most useful variables are as follows:

  • LLVM_TARGETS_TO_BUILD: LLVM supports code generation for different CPU architectures. By default, all these targets are built. Use this variable to specify the list of targets to build, separated by semicolons. The current targets are AArch64, AMDGPU, ARM, BPF, Hexagon, Lanai, Mips, MSP430, NVPTX, PowerPC, RISCV, Sparc, SystemZ, WebAssembly, X86, and XCore. all can be used as shorthand for all targets. The names are case-sensitive. To only enable PowerPC and the System Z target, you must specify -DLLVM_TARGETS_TO_BUILD="PowerPC;SystemZ".
  • LLVM_ENABLE_PROJECTS: This is a list of the projects you want to build, separated by semicolons. The source for the projects must be at the same level as the llvm directory (side-by-side layout). The current list is clang, clang-tools-extra, compiler-rt, debuginfo-tests, lib, libclc, libcxx, libcxxabi, libunwind, lld, lldb, llgo, mlir, openmp, parallel-libs, polly, and pstl. all can be used as shorthand for all the projects in this list. To build Clang and llgo together with LLVM, you must specify -DLLVM_ENABLE_PROJECT="clang;llgo".
  • LLVM_ENABLE_ASSERTIONS: If set to ON, then assertion checks are enabled. These checks help find errors and are very useful during development. The default value is ON for a DEBUG build and OFF otherwise. To turn assertion checks on (for example, for a RELEASE build), you must specify –DLLVM_ENABLE_ASSERTIONS=ON.
  • LLVM_ENABLE_EXPENSIVE_CHECKS: This enables some expensive checks that can really slow down your compilation speed or consume large amounts of memory. The default value is OFF. To turn these checks on, you must specify -DLLVM_ENABLE_EXPENSIVE_CHECKS=ON.
  • LLVM_APPEND_VC_REV: LLVM tools such as llc display the LLVM version they are based on, besides other information if the–version command-line option is provided. This version information is based on the LLVM_REVISION C macro. By default, not only the LLVM version but also the Git hash of the latest commit is part of the version information. This is handy in case you are following the development of the master branch because it makes it clear which Git commit the tool is based on. If this isn't required, then this can be turned off with –DLLVM_APPEND_VC_REV=OFF.
  • LLVM_ENABLE_THREADS: LLVM automatically includes thread support if a threading library is detected (usually, the pthreads library). Furthermore, in this case, LLVM assumes that the compiler supports thread-local storage (TLS). If you don't want thread support or your compiler does not support TLS, then you can turn it off with -DLLVM_ENABLE_THREADS=OFF.
  • LLVM_ENABLE_EH: The LLVM projects do not use C++ exception handling, so they turn exception support off by default. This setting can be incompatible with other libraries your project is linking with. If needed, you can enable exception support by specifying –DLLVM_ENABLE_EH=ON.
  • LLVM_ENABLE_RTTI: LVM uses a lightweight, self-built system for runtime type information. Generating C++ RTTI is turned off by default. Like the exception handling support, this may be incompatible with other libraries. To turn generation for C++ RTTI on, you must specify –DLLVM_ENABLE_RTTI=ON.
  • LLVM_ENABLE_WARNINGS: Compiling LLVM should generate no warning messages if possible. Due to this, the option to print warning messages is turned on by default. To turn it off, you must specify –DLLVM_ENABLE_WARNINGS=OFF.
  • LLVM_ENABLE_PEDANTIC: The LLVM source should be C/C++ language standard-conforming; hence, pedantic checking of the source is enabled by default. If possible, compiler-specific extensions are also disabled. To reverse this setting, you must specify –DLLVM_ENABLE_PEDANTIC=OFF.
  • LLVM_ENABLE_WERROR: If set to ON, then all the warnings are treated as errors – the compilation aborts as soon as warnings are found. It helps to find all the remaining warnings in the source. By default, it is turned off. To turn it on, you must specify –DLLVM_ENABLE_WERROR=ON.
  • LLVM_OPTIMIZED_TABLEGEN: Usually, the tablegen tool is built with the same options as the other parts of LLVM. At the same time, tablegen is used to generate large parts of the code generator. As a result, tablegen is much slower in a debug build, thus increasing the compile time noticeably. If this option is set to ON, then tablegen is compiled with optimization turned on, even for a debug build, possibly reducing compile time. The default is OFF. To turn this on, you must specify –DLLVM_OPTIMIZED_TABLEGEN=ON.
  • LLVM_USE_SPLIT_DWARF: If the build compiler is gcc or Clang, then turning on this option will instruct the compiler to generate the DWARF debug information in a separate file. The reduced size of the object files reduces the link time of debug builds significantly. The default is OFF. To turn this on, you must specify -LLVM_USE_SPLIT_DWARF=ON.

LLVM defines many more CMake variables. You can find the complete list in the LLVM documentation of CMake (https://releases.llvm.org/12.0.0/docs/CMake.html#llvm-specific-variables). The preceding list only contains the ones you are likely to need.