Cutting down building resources with better tooling
As we mentioned at the beginning of this chapter, if you build LLVM with the default (CMake) configurations, by invoking CMake and building the project in the following way, there is a high chance that the whole process will take hours to finish:
$ cmake ../llvm $ make all
This can be avoided by simply using better tools and changing some environments. In this section, we will cover some guidelines to help you choose the right tools and configurations that can both speed up your building time and improve memory footprints.
Replacing GNU Make with Ninja
The first improvement we can do is using the Ninja build tool (https://ninja-build.org) rather than GNU Make, which is the default build system generated by CMake on major Linux/Unix platforms.
Here are the steps you can use to set up Ninja on your system:
- On Ubuntu, for example, you can install Ninja by using this command:
$ sudo apt install ninja-build
Ninja is also available in most Linux distributions.
- Then, when you're invoking CMake for your LLVM build, add an extra argument:
$ cmake -G "Ninja" ../llvm
- Finally, use the following build command instead:
$ ninja all
Ninja runs significantly faster than GNU Make on large code bases such as LLVM. One of the secrets behind Ninja's blazing fast running speed is that while the majority of build scripts such as Makefile are designed to be written manually, the syntax of Ninja's build script, build.ninja, is more similar to assembly code, which should not be edited by developers but generated by other higher-level build systems such as CMake. The fact that Ninja uses an assembly-like build script allows it to do many optimizations under the hood and get rid of many redundancies, such as slower parsing speeds, when invoking the build. Ninja also has a good reputation for generating better dependencies among build targets.
Ninja makes clever decisions in terms of its degree of parallelization; that is, how many jobs you want to execute in parallel. So, usually, you don't need to worry about this. If you want to explicitly assign the number of worker threads, the same command-line option used by GNU Make still works here:
$ ninja -j8 all
Let's now see how you can avoid using the BFD linker.
Avoiding the use of the BFD linker
The second improvement we can do is using linkers other than the BFD linker, which is the default linker used in most Linux systems. The BFD linker, despite being the most mature linker on Unix/Linux systems, is not optimized for speed or memory consumption. This would create a performance bottleneck, especially for large projects such as LLVM. This is because, unlike the compiling phase, it's pretty hard for the linking phase to do file-level parallelization. Not to mention the fact that the BFD linker's peak memory consumption when building LLVM usually takes about 20 GB, causing a burden on computers with small amounts of memory. Fortunately, there are at least two linkers in the wild that provide both good single-thread performance and low memory consumption: the GNU gold linker and LLVM's own linker, LLD.
The gold linker was originally developed by Google and donated to GNU's binutils. You should have it sitting in the binutils package by default in modern Linux distributions. LLD is one of LLVM's subprojects with even faster linking speed and an experimental parallel linking technique. Some of the Linux distributions (newer Ubuntu versions, for example) already have LLD in their package repository. You can also download the prebuilt version from LLVM's official website.
To use the gold linker or LLD to build your LLVM source tree, add an extra CMake argument with the name of the linker you want to use.
For the gold linker, use the following command:
$ cmake -G "Ninja" -DLLVM_USE_LINKER=gold ../llvm
Similarly, for LLD, use the following command:
$ cmake -G "Ninja" -DLLVM_USE_LINKER=lld ../llvm
Limiting the number of parallel threads for Linking
Limiting the number of parallel threads for linking is another way to reduce (peak) memory consumption. You can achieve this by assigning the LLVM_PARALLEL_LINK_JOBS=<N> CMake variable, where N is the desired number of working threads.
With that, we've learned that by simply using different tools, the building time could be reduced significantly. In the next section, we're going to improve this building speed by tweaking LLVM's CMake arguments.
