The Digital Mars D compiler

DMD is the reference compiler for the D programming language. Created by Walter Bright, it is still maintained by him with the help of a handful of talented volunteers. It's not the only D compiler out there. GDC is built on top of the GNU Compiler Collection (GCC) and LDC uses the LLVM toolchain. Both compilers were created, and continue to be maintained, by members of the D community. As you write more D code, you'll find that DMD has blazingly fast compile times, while GDC and LDC produce highly optimized executables that tend to have better performance (though they are far from slow in terms of compilation speed). Because of this, it's not uncommon for D programmers to use DMD during development of a new project to take advantage of the faster compile times, then use one of the other compilers to build the final release and benefit from the more advanced optimizations. That said, we're going to be focused exclusively on DMD in this book. The code snippets and the sample project should compile with GDC or LDC just fine, but any compiler-specific instructions in the text will be for DMD.

There is nothing complex about installing DMD. Before we make that happen, it's important to understand a couple of things about how DMD works.

Frontends, backends, and linkers

One of the primary goals Walter established during D's early development was that it must be binary-compatible with C. Essentially, this means that it should be possible to compile a C source file with a C compiler and combine the output into a program that is written in D and compiled with a D compiler, or vice versa. To achieve this goal, output from a D compiler has to be in a format that C toolchains can understand. An in-depth discussion of compiler technology is quite a long way beyond the scope of this book, but it's necessary to have a minimal understanding of a small part of it in order to make DMD work for you more effectively.

Compilers typically have two major components that work together to create the final output: the frontend and the backend. The frontend is tied directly to a specific language. It takes source code as input and transforms it into an intermediate, language-agnostic format as output. The backend is tied to a specific platform. It takes the transformed code from the frontend as input and generates machine code as output, typically in the form of object files. Once the object files are created, ultimately one of two things will happen: either they are passed to a tool that links them into a final executable or they are passed to a tool that packs them into a library. The former tool is called a linker and the latter is called a librarian or archiver, though it's possible for one tool to perform both tasks.

For Walter to achieve his goal of binary compatibility with C, he opted to make use of existing toolchains where possible. This way, he could expend most of his effort on the frontend, the part that actually deals with D source code, and leave the rest to existing, tried-and-tested tools. He was already in the C and C++ compiler business—the company he owns, Digital Mars, distributes the Digital Mars C and C++ Compiler (DMC)—so he had his own existing backend and linker sitting on his hard drive. Appropriately, he began to implement a D frontend and hooked it up with the DMC backend to create DMD. Since the Digital Mars C(++) toolchain is Windows-specific, other options needed to be considered when it came time to port DMD to additional platforms. Walter's solution was to modify the backend to generate the appropriate output for each target platform, then have the compiler make use of each platform's system linker and librarian to generate the final binaries.

Running DMD on POSIX systems is a straightforward process. Since the compiler uses the system toolchain to create the final output on each system it supports, things tend to work smoothly without any conflicts. This includes support for both 32- and 64-bit output. The story on Windows isn't quite so rosy.

Note

In this book, POSIX is used to describe any non-Windows, Unix-like system supported by DMD, whether they are 100 percent POSIX-certified or not. That includes Linux, Mac OS X, and the various BSDs.

The problem is that the Digital Mars linker, OPTLINK, is ancient. It uses an object file format that is incompatible with most modern C and C++ toolchains. We'll explore this issue in more detail in Chapter 9, Connecting D with C, because it's an important thing to understand when interacting with C. Another point about OPTLINK is that it only supports 32-bit output. 64-bit support was implemented on Windows by giving DMD the ability to generate object files in a format understood by the Microsoft toolchain.

The major upside of this approach is that it eliminates the annoyances that come from conflicting object file formats on Windows, at least when compiling 64-bit binaries with the -m64 command line switch. The downside is that compiling 64-bit binaries with DMD now requires the installation either of a version of Windows SDK that includes the MS tools, or a non-Express version of Microsoft Visual Studio. DMD cannot be a completely self-contained distribution for 64-bit development on Windows. As of DMD 2.067, the Microsoft tools can also be used in place of the default Digital Mars toolchain to generate 32-bit binaries by passing a command line switch, -m32mscoff, to DMD.

Installing DMD

Installing the very latest release of DMD is always dead easy. Simply point your browser to http://dlang.org/download.html and pick your poison. Several options are available, including an installer for Windows, a DMG file for Mac OS X, deb packages for Ubuntu and Debian, RPM files for Fedora, CentOS and openSuse, and ZIP files for each of those platforms plus FreeBSD.

DMD is also available from the package repositories of most Linux systems, as well as through homebrew and macports for OS X users and the ports tree for BSD systems. If it isn't available in your platform's package repository, or if it's outdated, choose the appropriate package for your platform from the download page. The ZIP file is a good choice when you want more control over where the compiler is installed. Here, we're going to note a couple of specific points related to the Windows installer, then we'll see how to install from the ZIP file.

The Windows installer

Either a version of the Windows SDK that ships with the Microsoft linker or a non-Express version of Microsoft Visual Studio must be installed in order to compile 64-bit apps (or to use the -m32mscoff option) with DMD on Windows. It will help a great deal if one of these is installed before installing DMD. This way, the DMD installer can automatically find where the MS tools are located and configure the compiler to use them. If DMD is installed first, then its configuration file, sc.ini, must be manually edited for it to find them.

Note

Older versions of the Windows SDK shipped with a complete compiler toolchain. Since the release of the Windows 8 SDK, that is no longer the case. The Express versions of Visual Studio include the toolchain, but do not ship with the system libraries needed by DMD. Installing the Windows 8, 8.1, or 10 SDKs with a version of Visual Studio Express will provide everything necessary. When using an older version of the SDK, no additional software needs to be installed.

A better option is to install a non-Express version of Visual Studio. The Community versions are available for free. These include everything DMD needs in one package. Support for VS 2015 was added in DMD 2.069.

The DMD installer will also ask if you want to install Visual D. This is a Visual Studio plugin developed by Rainer Schuetze. While I strongly recommend that you use a text editor and the command line to work through the code samples in this book in order to better familiarize yourself with DMD, you might find Visual D quite useful later on, especially if you enjoy using Visual Studio.

Installing from the ZIP

Installing from the ZIP file isn't difficult, but requires a bit of manual manipulation for things to work easily from the command line. The required steps differ across platforms. We'll start with how to install from the zip on Windows before looking at installation on the supported POSIX platforms.

The Windows ZIP

Unzip the files into any directory. I recommend something short and easy to remember, such as C:\D (this is the default location used by the installer). Optionally, add the windows\bin subdirectory to the system PATH. Assuming you've unzipped in C:\D, the full path would be C:\D\dmd2\windows\bin.

The Windows version of DMD is distributed as a 32-bit program and can compile both 32-bit and 64-bit binaries. By default, it is configured to compile 32-bit using OPTLINK. If you're happy with that, there's nothing else to do. 64-bit binaries can be compiled by passing -m64 on the command line when invoking DMD but, as described, the MS toolchain will have to be installed for it to work. After the MS tools are installed, open the C:\D\dmd2\windows\bin\sc.ini file in a text editor. There, you'll find a block of instructions on how to configure DMD to use the Microsoft tools. Several of the standard Windows SDK and Visual Studio paths are already preconfigured, with most of them commented out. If the default, uncommented paths do not match your version of the Microsoft toolchain, it should be enough to comment out those lines by adding a semicolon, ;, in front of each and uncommenting the lines that match your installation by deleting the existing semicolons from in front of them. At this point, I'd like to remind you that the DMD installer takes care of all of this for you.

Note

On every supported platform except Windows, the DMD configuration file is called dmd.conf. On Windows, it's called sc.ini to maintain compatibility with the Digital Mars toolchain that DMD uses by default.

The POSIX ZIPs

On the three POSIX systems supported by DMD (Mac OS X, Linux, and FreeBSD), it's not uncommon to extract the ZIP in the home directory. This will result in a ~/dmd2 directory. The easiest thing to do with this is to leave the binaries where they are and add the appropriate subdirectory to the system PATH variable.

The Linux ZIP file contains both 32-bit and 64-bit versions of DMD; the Mac OS X ZIP file contains a 64-bit version of DMD; for FreeBSD, there are two ZIP files, one for 32-bit and one for 64-bit DMD. Choose the appropriate path from among the following to add to your system PATH variable:

• ~/dmd2/linux/bin32
• ~/dmd2/linux/bin64
• ~/dmd2/osx/bin
• ~/dmd2/freebsd/bin32
• ~/dmd2/freebsd/bin64

A 32-bit DMD will compile 32-bit apps by default, and a 64-bit DMD will compile 64-bit by default. For all versions of DMD, the command line flag -m32 will force 32-bit compilation and -m64 will force 64-bit compilation.

Compiling the example

With all of that background out of the way, it's time to get down to business. Open a command prompt and navigate to the directory where you saved the example. If you followed my advice, that should be $LEARNINGD/Chapter01. Now execute the following command:

dmd

If the PATH variable is properly configured, you should see the compiler print a header containing copyright information and the compiler version, followed by a list of supported command line options and a brief description of each (you might need to scroll up to see all of it). Now try this:

dmd hello.d

If you see errors or are unable to get DMD to run, head to the digitalmars.D.learn forum introduced earlier and ask for help. Otherwise, you should find yourself the proud parent of a brand new executable binary, called either hello or hello.exe, in the current directory. This is what its output looks like for me from the Windows Command Prompt:

You've now successfully compiled and executed a D program. It took you two commands to do so, but it doesn't have to be that way. DMD has a number of useful command line options, some of which are tied closely to language features. Other options are independent of language features and instead control the location or format of the final output, or instruct the compiler to perform a specific task. One such option, -run, is useful for trying out short examples like the ones you'll see in this book.

When DMD is executed with the -run option, it creates an executable in the current directory as normal. The difference is that -run causes the compiler to immediately execute the program and then delete the executable after the program exits. Try it on the hello.d example and see. This will save you from the execution step every time you want to compile and run an example.

Documentation and source code

In addition to several useful tools, the DMD distribution ships with some HTML documentation and the source code for the compiler, DRuntime, and Phobos. Both docs and source can be beneficial even after becoming familiar with D, but especially so while learning it.

The documentation

In the root directory where you installed or unzipped DMD, you'll find a subdirectory called html. Open the html/d/index.html file in a web browser and you'll see the front page of the documentation for your version of the compiler. The documentation for the latest compiler release is also available online at http://dlang.org/. In the navigation bar on the left-hand side of the page, as seen in the following screenshot, look for the links named D Reference and Standard Library; the former is the place to look up information about language features, the latter is the documentation for the DRuntime and Phobos libraries:

Now and again, you might find that the documentation for a function or a language feature isn't clear. Sometimes, it isn't as descriptive as it could be, or refers to functionality that has changed. Keep in mind that D is a community-driven language that relies on users to help by reporting problems or contributing fixes. If you'd like to contribute, documentation fixes can be submitted as pull requests at https://github.com/D-Programming-Language/dlang.org, and bug reports can be filed at https://issues.dlang.org/ under the websites component. Once that's out of the way, you still need to solve the problem that caused you to look in the docs in the first place. You can do that by visiting the forums and asking for help, or often by going right to the source.

The source

At the same level as the html directory lives another directory labelled src. Open it up and you'll see subdirectories called dmd, druntime, and phobos. In each is the source code for the eponymous project. I don't expect readers of this book to start modifying the compiler just yet, but you might want to read the files src/dmd/readme.txt and src/dmd/backendlicense.txt to understand the licenses of the DMD frontend and backend source. More immediately useful is the source for DRuntime and Phobos, which is all released under the Boost Software License.

As you're learning D, taking a peek at a function's implementation can help clarify its usage if the documentation comes up short. Later, studying the Phobos source can be a great way to learn some D idioms and other useful tricks. At some point in your D career, you might be interested in exploring what DRuntime does under the hood to manage your program. In the beginning, you'll likely find more benefit from the source in the core package. core is the primary DRuntime package, the part of the runtime that users directly interact with. We won't spend a lot of time on core in this book, but there will be brief discussions about a few of the modules it exposes as the need arises.

The source that ships with each DMD release is the source for that specific version of the compiler. Once a new version is out the door, development continues toward the next release. All such development happens at GitHub, specifically in the D Programming Language organization at https://github.com/D-Programming-Language. Here, you can find projects for the compiler, Phobos, Druntime, the website, and several tools and utilities, including the Windows installer, Visual D, and DUB (which we'll look at shortly). If you enjoy contributing to open source projects and also enjoy using D, there are many ways to contribute aside from committing your own source modifications. The quickest way to get started is to report bugs at https://issues.dlang.org/, or to review contributions at GitHub. The more users contribute, the faster the language moves forward.