Building Julia from sources on Linux

Building Julia allows you to test the latest developments and includes bug fixes. Moreover, when Julia is compiled, it is optimized for the hardware that the compilation is performed on. Consequently, building Julia from source code is the recommended option for those production environments where performance is strongly affected by platform-specific features. These instructions will also be valuable for those Julia users who would like to check out and experiment with the latest source versions from the Julia Git repository. 

In the following examples, we show how to install and build a stable version of Julia 1.0.1.

Getting ready

All the following examples have been tested on Ubuntu 18.04.1 LTS.

Here is a list of steps to be followed:

1. Open the console and install all the prerequisites. Please refer to the following script (run each shell command shown as follows):
   

$ sudo apt update
$ sudo apt install --yes build-essential python-minimal gfortran m4 cmake pkg-config libssl-dev 

2. Download the source code (run each shell command shown as follows; we assume that the commands are run in your home folder):

$ git clone git://github.com/JuliaLang/julia.git
$ cd julia
$ git checkout v1.0.1

In the GitHub repository for this recipe you will find the commands.txt file that contains the presented sequence of shell commands. 

How to do it...

In this section, we describe how to build Julia in three particular variations:

• With open source mathematical libraries
• With Intel's Math Kernel Library (MKL), but without Intel LIBM (Math Library)—this scenario requires registration on Intel's website
• With Intel's Math Kernel Library (MKL) and with Intel LIBM (Math Library)—a commercial license from Intel is required

The libraries from Intel (MKL and LIBM) provide an implementation for a number of mathematical operations optimized for Intel's processor architecture. In particular, the Intel MKL library contains optimized routines for BLAS, LAPACK, ScaLAPACK, sparse solvers, fast Fourier transforms, and vector math (for more details see https://software.intel.com/en-us/mkl). On the other hand, the Intel LIBM library provides highly optimized scalar math functions that serve as direct replacements for the standard C calls—this includes optimized versions of standard math library functions, such as exp, log, sin, and cos). More information on Intel LIBM can be found at https://software.intel.com/en-us/articles/implement-the-libm-math-library.

Before running each of the recipes, please make sure that you are inside the folder where you ran the checkout command for Julia (see the Getting ready section).

Option 1 - build Julia without Intel's MKL

Once you have followed the steps in the Getting ready section and Julia has been checked out from the Git repository, perform the following steps:

1. In order to build Julia, simply run the following bash shell command:

$ make -j $((`nproc`-1)) 1>build_log.txt 2>build_error.txt

The build logs will be available in the build_log.txt and build_error.txt files.

2. Once the Julia environment has been built, you can run the ./julia command and use versioninfo() to check your installation:

julia> versioninfo()
Julia Version 1.0.1
Commit 0d713926f8* (2018-09-29 19:05 UTC)
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: Intel(R) Xeon(R) Platinum 8124M CPU @ 3.00GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-6.0.0 (ORCJIT, skylake)

Option 2 - build Julia with Intel MKL (without Intel LIBM)

Once you have followed the steps in the Getting ready section and Julia has been checked out from the Git repository, perform the following the steps:

1. The MKL library is freely available from Intel. In order to get access to Intel MKL, you need to submit a form on Intel's website at https://software.intel.com/en-us/mkl.
2. Once the form has been completed, you receive an MKL download link (please note that the actual filenames may be different in the library version you obtain).

3. Execute the following commands to install MKL:

$ cd ~
# Get link from MKL website
$ wget http://registrationcenter-download.intel.com/[go to Intel MKL web site to get link]/l_mkl_2019.0.117.tgz
$ tar zxvf l_mkl_2019.0.117.tgz
$ cd l_mkl_2019.0.117
$ sudo bash install.sh

4. Once the Intel MKL library is installed, you can build Julia (run each shell command as shown):

cd ~/julia
echo "USEICC = 0" >> Make.user
echo "USEIFC = 0" >> Make.user
echo "USE_INTEL_MKL = 1" >> Make.user
echo "USE_INTEL_LIBM = 0" >> Make.user
​
source /opt/intel/bin/compilervars.sh intel64
​
make -j $((`nproc`-1)) 1>build_log.txt 2>build_error.txt

The build logs will be available in the build_log.txt and build_error.txt files.

5. Once Julia is successfully built, run the ./julia command to start it.
6. Use versioninfo() to check the status of Julia. Information about the MKL status is available from the ENV["MKL_INTERFACE_LAYER"] system variable. Take a look at a sample screen, as follows:

julia> versioninfo()
Julia Version 1.0.1
Commit 0d713926f8* (2018-09-29 19:05 UTC)
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: Intel(R) Xeon(R) Platinum 8124M CPU @ 3.00GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-6.0.0 (ORCJIT, skylake)

julia> ENV["MKL_INTERFACE_LAYER"]
"ILP64"

Please note that if you build Julia with MKL, whenever you start a new Linux session you need to tell Julia where the Intel compilers are so that it can properly find and use the MKL libraries. This is done by executing the following bash command:

$ source /opt/intel/bin/compilervars.sh intel64

The preceding command needs to be executed each time prior to launching the julia process in a new environment.

Option 3 - build Julia with Intel MKL and with Intel LIBM

Once you have followed the steps in the Getting ready section and Julia has been checked out from the Git repository, perform the following steps:

1. Acquire an Intel Parallel Studio XE (https://software.intel.com/en-us/c-compilers/ipsxe) license, which entitles you to use the Intel C++ compilers (https://software.intel.com/en-us/c-compilers) along with the Intel Math Library (Intel LIBM, available at https://software.intel.com/en-us/node/522653).
   
2. Once you acquire the license, together with a download link from Intel, download and install the software by running the shell commands given as follows (please note that the actual filenames may be different in the library version you obtain):

$ cd ~
# Get the link from Intel C++ compilers website
$ wget http://[go to Intel to get link]/parallel_studio_xe_2018_update3_professional_edition.tgz
$ tar zxvf parallel_studio_xe_2018_update3_professional_edition.tgz
$ cd parallel_studio_xe_2018_update3_professional_edition
$ sudo bash install.sh

3. Select the MKL among the installation options in Intel Parallel Studio XE. 
4. Build Julia (run each shell command as shown):

cd ~/julia
echo "USEICC = 0" >> Make.user
echo "USEIFC = 0" >> Make.user
echo "USE_INTEL_MKL = 1" >> Make.user
echo "USE_INTEL_LIBM = 1" >> Make.user
​
source /opt/intel/bin/compilervars.sh intel64
​
make -j $((`nproc`-1)) 1>build_log.txt 2>build_error.txt

The build logs will be available in the build_log.txt and build_error.txt files.

5. Once Julia has been compiled, you can start it with the ./julia command and use the versioninfo() command—the LIBM parameter should point to libimf.

Please note that if you build Julia with MKL/LIBM, when you start a new Linux session, you need to tell Julia where Intel compilers are so it can properly find and use MKL/LIBM libraries:

$ source /opt/intel/bin/compilervars.sh intel64

The preceding command needs to be executed each time before launching the julia process in a new environment (hence, one might want to add that command to the ~/.profile start-up file).

How it works...

For the highest performance, it is recommended to compile Julia with the Linux Intel MKL (Intel MKL is available at https://software.intel.com/en-us/mkl) drivers, which are available for free from Intel. The numerical performance can also be enhanced by using the Intel Math Library (Intel LIBM is available at https://software.intel.com/en-us/node/522653). However, LIBM can only be obtained with the Intel C++ compilers (see https://software.intel.com/en-us/c-compilers), which are free for academic and open source use but paid otherwise. Therefore, some users might be interested in building Julia with MKL, but without LIBM.

Please note that in all the scenarios outlined, we use GNU compilers rather than Intel compilers—even when using Intel's MKL and LIBM libraries. If you want to use Intel's compilers, you need to set the appropriate options in the Make.user file (that is, change USEICC = 0 and USEIFC = 0 to USEICC = 1 and USEIFC = 1). However, Intel compilers are currently not supported by Julia compiler scripts (see https://github.com/JuliaLang/julia/issues/23407).

There's more...

Once Julia is installed on your system, it can be run by giving the full path to the Julia executable (for example, ~/julia/julia). This is not always convenient—many users simply want to type julia to get Julia running:

$ sudo ln -s /home/ubuntu/julia/usr/bin/julia /usr/local/bin/julia

The preceding path assumes that you installed Julia as the ubuntu user in your home folder. If you have installed Julia to a different location, please update the path accordingly.

See also

If you want to build Julia in a supercomputing environment (for example, Cray), please follow the online tutorial written by one of the authors of this book, available at https://github.com/pszufe/Building_Julia_On_Cray_and_Clusters.