DUB – the D build tool and package manager

There is nothing challenging about compiling simple D programs with DMD on the command line, but things can get a bit out of hand as a project grows. A complex D program will be composed of numerous modules and can be dependent on several third-party libraries. There are a number of approaches that users have taken to compile their D projects. Some use makefiles or existing tools that have some form of support for D, such as CMake or SCons, and a few have even created their own build tools from scratch. For several years, I used a custom build script written in D for most of my projects.

In the past, having so many different approaches for building D code could sometimes be a bit of an annoyance for users of D libraries, particularly when using multiple libraries that each had a different build system. Users of a library I maintain often asked me to add support for different build systems to better match their own workflow, something I was extremely reluctant to do since learning, implementing, and maintaining such support would cut into my already scarce free time. Thankfully, this is one growing pain that the D community has left behind. In this section, I'm going to introduce you to DUB, which has fast become a central part of the D ecosystem.

Created by Sönke Ludwig, DUB provides a unified approach to building D projects. To facilitate this, it also serves as a distribution platform. Anyone with a D library can share it with the world by creating a DUB configuration for it and registering it in the DUB registry. Subsequently, any DUB user can use the library by adding a single line to her project's DUB configuration file. It can also be used to easily distribute open source executables. Potential users can use one DUB command to download, compile, and run a D program.

In Chapter 8, Exploring the Wide World of D, we're going to look at the DUB registry and see how to use DUB-enabled libraries in your own projects as we take a tour of the D ecosystem. Now, we're going to set up a DUB configuration specifically for MovieMan, but the same steps can be applied to any new project you create.

Getting started

DUB is available in most of the common package repositories and the latest release can always be found at http://code.dlang.org/download. Download the appropriate tarball, installer, or ZIP file for your platform. Installing from ZIP or tarball generally requires no special steps beyond unzipping the files to a directory of your choice (such as C:\dub or ~\dub) and ensuring the directory is on your PATH. POSIX systems will also need to have some prerequisites installed, so be sure to read the instructions on the download page. The Windows installer and ZIP both come bundled with all dependencies.

Once the program is installed and PATH is properly configured, open a command prompt and type the following command:

dub -h

If all goes well, you'll see a list of commands and command-line options (you will likely have to scroll up to see the whole thing). The command-line options are generic, meaning they can be passed to dub with any DUB command.

DUB usage revolves around commands. The default behavior when no commands are specified is to build and run any DUB project in the current directory. Unlike command-line options, DUB commands have no preceding dash. You can get help on any DUB command by passing its name to dub followed by -h or --help (the former is a synonym for the latter). For example, to get help with the build command, type the following:

dub build -h

This will print a brief description of the command, along with any command-specific arguments it accepts, followed by the same list of common command-line options you saw with dub -h. Much of what you need to know to get started with DUB on the command line can be gleaned from using the -h option. If you get stuck, you can ask for help in the DUB forums at http://forum.rejectedsoftware.com/groups/rejectedsoftware.dub/.

Configuring the MovieMan project

It's possible to set up a new project by hand, but DUB can set up a common directory structure and an initial configuration with a simple command called init. By default, with no arguments, init will create a new project using the current working directory as the project root and the name of that directory (not the full path) as the project name. If an argument is given to init, DUB will use it as the project name and create a new subdirectory of the same name in the current directory. Let's do that now for MovieMan.

Navigate to $LEARNINGD/Chapter01 and execute the following command:

dub init MovieMan

Listing the contents of the MovieMan directory shows the following:

09/20/2015  12:31 PM    <DIR>          .
09/20/2015  12:31 PM    <DIR>          ..
09/20/2015  12:31 PM                38 .gitignore
09/20/2015  12:31 PM               120 dub.sdl
09/20/2015  12:31 PM    <DIR>          source

DUB has generated three files and one subdirectory in the project directory tree. You can see two of the files in the preceding output. The third file, app.d, is inside the source subdirectory.

DUB is tightly integrated with Git, a distributed source control system. The .gitignore file is used to tell Git to pay no attention to files whose names match certain patterns. DUB is helping out by generating this file and including some file types that aren't commonly included in source control, but that might be created as part of the build process. For our purposes, we can ignore .gitignore.

By default, when running DUB in a project directory, it searches for the project source files in the source subdirectory. Also by default, it expects to find the source/app.d file. If this file exists, an executable will be built; if not, DUB compiles a library. The app.d that dub init generates for us isn't empty. It contains a main function that prints a line of text to the screen.

Sometimes, you might want to add DUB support to an existing project where it might be problematic to change the directory structure or rename the main module to app.d, or perhaps you just don't want to follow the convention that DUB expects. No matter your reasons, all of the defaults in the preceding paragraph can be overridden in the configuration file, dub.sdl. As you gain more experience with DUB, you'll learn how to go beyond the simple configurations we use in this book.

Understanding dub.sdl

DUB supports two formats for its project configuration files: Simple Declarative Language (SDLang) and JavaScript Object Notation (JSON). The former is the default format output by the init command (the command dub init -fjson will create dub.json instead). Originally, JSON was the only format supported. As of DUB 0.9.24, SDLang is the preferred format, though JSON will be supported indefinitely. Go to http://semitwist.com/sdl-mirror/Language+Guide.html for more about SDLang and http://json.org/ for JSON.

DUB requires a configuration file to be present in any project it is intended to manage. It supports a number of predefined fields for defining metadata that provide information about the project, such as its name and description, and instructions for DUB to follow when building the project. The documentation at http://code.dlang.org/package-format is the place to visit to keep informed on supported DUB configuration options.

Note

You've already seen the term package in reference to a hierarchical grouping of D modules. It's common to use the term DUB package to refer to a DUB project.

The configuration generated by the init command is entirely metadata. The metadata in a DUB package configuration comprises the first few lines by convention. None of the entries are required to be in any particular order. Take a look at the complete contents of the dub.sdl file generated by dub init for the MovieMan project:

name "movieman"
description "A minimal D application."
copyright "Copyright © 2015, Mike Parker" authors "Mike Parker"

Every DUB package must have a name that can be used as a unique identifier. The name should be comprised of lowercase alphanumeric characters. Dashes and underscores are also permitted. When generating a project, DUB will always use lowercase letters. If the configuration file is manually edited to include uppercase characters in the package name, DUB will internally convert them to lowercase when it loads the file.

The name field is the only one that is required. The other fields can be deleted and the project will build just fine. However, if there is any intention to register the package with the online DUB registry, then it's a good idea to always include the generated metadata fields at a minimum. There are other useful metadata fields that are not generated by default, such as license and homepage. The more metadata provided, the more potential users can learn about the project from the DUB registry.

Most of the metadata items are only visible from the package-specific page in the registry. The name and description fields are the only two that are displayed in the list of all registered packages. As such, the description shouldn't be very long or overly vague. What you want is a short, succinct summary of the package so that anyone browsing the registry can determine whether it's something he might be interested in. We aren't going to register MovieMan in the DUB registry, but if we were, then we might use something like the following to describe it:

description "A command-line DVD database."

The name used in the copyright and authors fields for a generated dub.json comes from the name of the user account under which dub init was executed. Multiple authors can be listed as follows:

authors "Bjarne Gosling" "Brian Pike"

Several configuration fields recognized by DUB require a list of values.

The majority of DUB directives are not simply metadata, but they directly influence the build process. There are directives for specifying library dependencies, compiler flags, which compiler to use, and more. We'll add one to the MovieMan configuration shortly.

Building and running MovieMan

DUB allows you to build a project and run the resulting executable in two separate steps, or to do it all at once. To build the project without running it, you use the aptly-named build command. To both build and execute the project, you can use the run command, which is the default behavior when invoking dub without specifying any commands (so both dub and dub run are the same thing).

Let's give DUB a go at building the project, first by specifying the build command. Here's what I see on my system:

C:\LearningD\Chapter01\MovieMan>dub build
Building movieman ~master configuration "application", build type debug.
Compiling using dmd...
Linking...

You can see that, by default, DUB isn't very verbose about what it's doing. It tells you which build configuration it's building and the build type, lets you know when it starts compiling and which compiler it's using, and then tells you that it's creating (linking) the executable. The build configuration, build type, and compiler can all be specified in the configuration file.

Now that the executable is built, it's just sitting there waiting to be executed. Let's make use of the run command to do so:

C:\LearningD\Chapter01\MovieMan>dub run
Target movieman ~master is up to date. Use --force to rebuild.
Running .\movieman.exe
Edit source/app.d to start your project.

If you haven't yet examined the generated app.d in your text editor, it might not be immediately obvious to you that the last line is the actual output of the generated code and the previous lines are from DUB itself. Look at the first line of output in particular. This is telling you that the executable is already built and that, if you want to build it again, you should use the --force option to do so. That might seem like an odd message to see when running the program. To understand it, try running the build command again:

C:\LearningD\Chapter01\MovieMan>dub build
Target movieman ~master is up to date. Use --force to rebuild.

As you can see, DUB refused to build the project and is telling you why and what to do. Try again, this time with the --force option and it will build successfully. Also, see what happens when you invoke dub and dub --force.

So the take away here is that DUB isn't going to rebuild your project if it is already up-to-date and it will always check whether a rebuild is needed before executing the program. We can define "up-to-date" to mean that the timestamp of the executable is more recent than the last modification time of any of the source files. If you edit any source file in the source directory, then trying to build again will succeed. To demonstrate that, let's edit app.d to look like the following:

import std.stdio;
void main() {
  writeln("Hi! My name is MovieMan.");
}

Now, executing either dub or dub run will cause DUB to notice that app.d has a more recent timestamp than the executable. In response, it will rebuild the application before executing it.

Tip

To build or not to build

The build command comes in handy when you invoke it as you add new code to a project. I like to do so after adding several new lines, or a new function, just to make sure everything still compiles. DMD is so fast that, even as a project gets larger, this does not impede development. Only when I've completed adding or modifying a feature do I invoke dub run to make sure the app is working as expected.

Changing the output directory

Executable programs often ship with a number of resources. These generally should be located either in the same directory as the executable or in subdirectories. By default, the binary that DUB builds, whether it is a library or an executable, is output to the same directory in which the dub.json file resides (which we can call the project or package root). When there are resources to worry about, this can look a little cluttered. Even without resources, I prefer my binaries to be written in their own directory. This can be accomplished with one simple addition to dub.sdl:

targetPath "bin"

targetPath is a build directive that tells DUB where to write the binary. It can be placed anywhere in the file, but convention dictates that it follow the metadata. Here, I've used a relative path that will cause DUB to create a subdirectory named bin (if it doesn't already exist) and write the binary there. Absolute paths, such as C:\dev\MovieMan or /~/bin/MovieMan, are also acceptable. However, you should prefer relative paths to keep your DUB packages completely self-contained if you are going to distribute them.