Data Visualization with D3.js Cookbook

Chapter 1. Getting Started with D3.js

In this chapter we will cover:

Setting up a simple D3 development environment
Setting up an NPM-based development environment
Understanding D3-style JavaScript

Introduction

This chapter is designed to get you up and running with D3.js, covering fundamental aspects, such as what D3.js is, and how to set up a typical D3.js data visualization environment. One particular section is also devoted in covering some lesser known areas of JavaScript that D3.js relies heavily on.

What is D3? D3 refers to Data-Driven Documents, and according to the official D3 Wiki:

D3.js is a JavaScript library for manipulating documents based on data. D3 helps you bring data to life using HTML, SVG, and CSS. D3's emphasis on web standards gives you the full capabilities of modern browsers without tying yourself to a proprietary framework, combining powerful visualization components and a data-driven approach to DOM manipulation.
D3 Wiki (2013, August)

In a sense, D3 is a specialized JavaScript library that allows you to create amazing data visualizations using a simpler (data driven) approach by leveraging existing web standards. D3.js was created by Mike Bostock (http://bost.ocks.org/mike/) and superseded his previous work on a different JavaScript data visualization library called Protovis. For more information on how D3 was created and on the theory that influenced both Protovis and D3.js, please check out links in the following information box. Here in this book we will focus more on how to use D3.js to power your visualization. Initially, some aspects of D3 may be a bit confusing due to its different approach to data visualization using JavaScript. I hope that over the course of this book, a large number of topics, both basic and advanced, will make you comfortable and effective with D3. Once properly understood, D3 can improve your productivity and expressiveness with data visualizations by orders of magnitude.

Note

For more formal introduction to the idea behind D3 see the Declarative Language Design for Interactive Visualization paper published by Mike Bostock on IEEE InfoVis 2010 http://vis.stanford.edu/papers/protovis-design.

If you are interested to know how D3 came about, I recommend you to check out the D3: Data-Driven Document paper published by Mike Bostock on IEEE InfoVis 2011 at http://vis.stanford.edu/papers/d3.

Protovis, the predecessor of D3.js, also created by Mike Bostock and Jeff Heer of the Stanford Visualization Group can be found at http://mbostock.github.io/protovis/.

Setting up a simple D3 development environment

First thing you need when starting a D3 powered data visualization project is a working development environment. In this recipe, we will show you how a simple D3 development environment can be set up within minutes.

Getting Ready

Before we start, make sure you have your favorite text editor installed and ready on your computer.

How to do it...

We'll start by downloading D3.js:

Download the latest stable version of D3.js from http://d3js.org/. You can download the archived, older releases from https://github.com/mbostock/d3/tags. Additionally, if you are interested in trying out the bleeding edge D3 build on master branch, then you can fork https://github.com/mbostock/d3.
Once downloaded and unzipped, you will find three files d3.v3.js, d3.v3.min.js, and its license in the extracted folder. For development it is recommended to use d3.v3.js, the "non-uglified" (minimized) version, since it can help you trace and debug JavaScript inside D3 library. Once extracted place the d3.v3.js file in the same folder with an index.html file containing the following HTML:
```

<!DOCTYPE html>
<html>
<head>
    <meta charset="utf-8">
    <title>Simple D3 Dev Env</title>
    <script type="text/javascript" src="d3.v3.js"></script>
</head>
<body>

</body>
</html>
```

Tip

If you download D3 from a source or a tagged version, the JavaScript file name will be slightly different. Instead of d3.v3.js, it will simply be called d3.js.

This is all you need to create, in its simplest form, a D3-powered data visualization development environment. With this setup you can essentially open the HTML file using your favorite text editor to start your development and also view your visualization by opening the file in your browser.

Note

The source code for this recipe can be found at https://github.com/NickQiZhu/d3-cookbook/tree/master/src/chapter1/simple-dev-env.

How it works...

D3 JavaScript library is very self-sufficient. It has no dependency on any JavaScript library than other what your browser already provides. In fact, it can even be used in a non-browser environment such as Node.js with some minimum setup (I will cover this in more detail in later chapters).

Tip

If your visualization's target browser environment includes Internet Explorer 9, it is recommended to use the compatibility library Aight, which can be found at https://github.com/shawnbot/aight, and Sizzle selector engine at http://sizzlejs.com/.

Having the following character encoding instruction in header section is critical:

    <meta charset="utf-8">

The character encoding instructs browsers and validators what set of characters to use when rendering web pages. Otherwise your browser will not be able to load D3 JavaScript library since D3 uses utf-8 character for certain symbols such as π.

Note

D3 is completely open source, and it is open sourced under a custom license agreement created by its author Michael Bostock. This license is pretty similar to the popular MIT license with only one exception where it explicitly states that Michael Bostock's name cannot be used to endorse or promote products derived from this software without permission.

There's more...

Throughout this cookbook numerous recipe code examples will be provided. All example source code are provided and hosted on GitHub (https://github.com/) a popular open source social coding repository platform.

How to get source code

The easiest way to get all the recipe source code that you need is to clone the Git repository (https://github.com/NickQiZhu/d3-cookbook) for this book. If you are not planning to set up a development environment for the recipes then you can safely skip this section.

Tip

If you are not familiar with Git, clone is similar to the check-out concept in other versions of control software. However cloning does a lot more than simply checking out the files. It also copies all branches and histories to your local machine effectively cloning the entire repository to your local machine so you can work completely offline with this cloned repository in your own environment.

First install a Git client on your computer. You can find a list of Git client software here http://git-scm.com/downloads, and a detailed guide on how to install it on different operating systems here http://git-scm.com/book/en/Getting-Started-Installing-Git.

Tip

Another popular way to get Git and GitHub working is to install the GitHub client, which gives you a richer set of features than simply Git. However, at the time of writing, GitHub only offered client software for Windows and Mac OS.

GitHub for Windows: http://windows.github.com/.

GitHub for Mac: http://mac.github.com/.

Once the Git client is installed, simply issuing the following command will download all recipe source code to your computer:

> git clone git://github.com/NickQiZhu/d3-cookbook.git

Tip

Or if you choose to use GitHub client, then simply click the Fork button on the repository page https://github.com/NickQiZhu/d3-cookbook. This will make this repository appear in your GitHub client.

Setting up an NPM-based development environment

When you are working on a more complex data visualization project that requires the use of a number of JavaScript libraries, the simple solution we discussed before might become a bit clumsy and unwieldy. In this section, we will demonstrate an improved setup using Node Packaged Modules (NPM)—a de facto JavaScript library repository management system. If you are as impatient as me and want to get to the meaty part of the book—the recipes—you can safely skip this section and come back when you need to set up a more production-ready environment for your project.

Getting Ready

Before we start please make sure you have NPM properly installed. NPM comes as part of the Node.js installation. You can download Node.js from http://nodejs.org/download/. Select the correct Node.js binary build for your OS. Once installed the npm command will become available in your terminal console.

> npm -v 
1.2.14

The preceding command prints out the version number of your NPM client indicating the installation is successful.

How to do it...

With NPM installed, now we can create a package descriptor file to automate some of the manual setup steps.

First, under your project folder, create a file named package.json containing the following code:

{
  "name": "d3-project-template",
  "version": "0.1.0",
  "description": "Ready to go d3 data visualization project template",
  "keywords": [
    "data visualization",
    "d3"
  ],
  "homepage": "<project home page>",
  "author": {
    "name": "<your name>",
    "url": "<your url>"
  },
  "repository": {
    "type": "git",
    "url": "<source repo url>"
  },
  "dependencies": {
      "d3":"3.x"
  },
  "devDependencies": {
      "uglify-js": "2.x"
  }
}

Once the package.json file is defined, you can simply run:
```
> npm install
```

How it works...

Most of the fields in the package.json file are for informational purpose only, such as the name, description, homepage, author, and the repository. The name and version field will be used if you decide to publish your library into an NPM repository in the future. What we really care about, at this point, is the dependencies and devDependencies fields.

The dependencies field describes the runtime library dependencies that your project has, meaning the libraries your project needs to run properly in a browser. In this simple example we only have one dependency on d3. d3 is the name that D3 library is published under in the NPM repository. The version number 3.x signifies that this project is compatible with any version 3 releases, and NPM should retrieve the latest stable version 3 build to satisfy this dependency.
Tip
D3 is a self-sufficient library with zero external runtime dependency. However, this does not mean that it cannot work with other popular JavaScript libraries. I regularly use D3 with other libraries to make my job easier, for example, JQuery, Zepto.js, Underscore.js, and Backbone.js.
The devDependencies field describes development time (compile time) library dependencies. What this means is that, libraries specified under this category are only required in order to build this project, and not required for running your JavaScript project.

Note

Detailed NPM package JSON file documentation can be found at https://npmjs.org/doc/json.html.

Executing the npm install command will automatically trigger NPM to download all dependencies that your project requires including your dependencies' dependencies recursively. All dependency libraries will be downloaded into node_modules folder under your project root folder. When this is done you can just simply create your HTML file as it has been shown in the previous recipe, and load your D3 JavaScript library directly from node_modules/d3/d3.js.

The source code for this recipe with an automated build script can be found at https://github.com/NickQiZhu/d3-cookbook/tree/master/src/chapter1/npm-dev-env.

Relying on NPM is a simple and yet effective way to save you from all the trouble of downloading JavaScript libraries manually and the constant need of keeping them up-to-date. However, an astute reader might have already noticed that with this power we can easily push our environment setup to the next level. Imagine if you are building a large visualization project where thousands of lines of JavaScript code will be created, obviously our simple setup described here is no longer sufficient. However modular JavaScript development by itself can fill an entire book; therefore we are not going to try to cover this topic since our focus is on data visualization and D3. If you are interested please refer the source code for this recipe where it is demonstrated how a more modular approach can be implemented on top of what we described here with a simple automated build script. In later chapters, when unit test related recipes are discussed, we will expand the coverage on this topic to show how our setup can be enhanced to run automated unit tests.

There's more...

Though in previous sections, it was mentioned that you can just open the HTML page that you have created using your browser to view your visualization result directly, this approach does have its limitations. This simple approach stops working once we need to load data from separate data file (this is what we will do in later chapters and it is also the most likely case in your daily working environment) due to the browser's built-in security policy. To get around this security constraint it is highly recommended that you set up a local HTTP server so your HTML page and the data file can be served from this server instead of loaded from file system directly.

Setup a local HTTP server

There are probably a dozen ways to set up an HTTP server on your computer based on which operating system you use and which software package you decide to use to act as an HTTP server. Here I will attempt to cover some of the most popular setups.

Python Simple HTTP Server

This is my favorite for development and fast prototyping. If you have Python installed on your OS, which is usually the case with any Unix/Linux/Mac OS distro, then you can simply type this command in your terminal:

> python –m SimpleHTTPServer 8888

Or with newer Python distribution:

> python –m http.server

This little python program will launch an HTTP server and start serving any file right from the folder where this program is launched. This is by far the easiest way to get an HTTP server running on any OS.

Note

If you don't have python installed on your computer yet, you can get it from http://www.python.org/getit/. It works on all modern OS including Windows, Linux and Mac.

Node.js HTTP Server

If you have Node.js installed, perhaps as part of the development environment setup exercise we did in the previous section, then you can simply install the http-server module. Similar to Python Simple HTTP Server, this module will allow you to launch a lightweight HTTP server from any folder and starting serving pages right away.

First install the http-server module:

> npm install http-server –g

The -g option in this command will install http-server module globally so it will become available in your command line terminal automatically. Once this is done, then you can launch the server from any folder you are in by simply issuing the following command:

> http-server .

This command will launch a Node.js powered HTTP server on the default port 8080 or if you want you can use the –p option to provide a custom port number for it.

Tip

If you are running the npm install command on Linux/Unix/Mac OS, you will need to run the command in sudo mode or as root in order to use the –g global installation option.

Understanding D3-style JavaScript

D3 is designed and built using functional style JavaScript which might come as to seem unfamiliar or even alien to someone who is more comfortable with the procedural or object-oriented JavaScript styles. This recipe is designed to cover some of the most fundamental concepts in functional JavaScript required to make sense of D3, and furthermore enable you to write your visualization code in D3 style.

Getting ready

Open your local copy of the following file in your web browser: https://github.com/NickQiZhu/d3-cookbook/blob/master/src/chapter1/functional-js.html

How to do it...

Let's dig a little deeper into the good part of JavaScript—the more functional side. Take a look at the following code snippet:

function SimpleWidget(spec) {
  var instance = {}; // <-- A

  var headline, description; // <-- B

  instance.render = function () {
    var div = d3.select('body').append("div");

    div.append("h3").text(headline); // <-- C

    div.attr("class", "box")
    .attr("style", "color:" + spec.color) // <-- D
      .append("p")
      .text(description); // <-- E

    return instance; // <-- F
  };

  instance.headline = function (h) {
    if (!arguments.length) h; // <-- G
      headline = h;
    return instance; // <-- H
  };

  instance.description = function (d) {
    if (!arguments.length) d;
      description = d;
    return instance;
  };

  return instance; // <-- I
}

  var widget = SimpleWidget({color: "#6495ed"})
    .headline("Simple Widget")
    .description("This is a simple widget demonstrating functional javascript.");
  widget.render();

This code snippet generates the following simple widget on your web page:

A Simple Widget with functional JavaScript

How it works...

Despite its simplicity, the interface of this widget has this undeniable similarity to D3 style of JavaScript. This is not by coincidence but rather by leveraging a JavaScript programming paradigm called functional objects. Like many interesting topics, this is another topic that can fill an entire book by itself; nevertheless I will try to cover the most important and useful aspects of this particular paradigm in this section so you the reader cannot only understand D3's syntax but will also be able to create a library in this fashion. As stated on D3's project Wiki this functional programming style gives D3 much of its flexibility:

D3's functional style allows code reuse through a diverse collection of components and plugins.
D3 Wiki (2013, August)

Functions are objects

Functions in JavaScript are objects. Like any other object, function is just a collection of name and value pair. The only difference between a function object and a regular object is that function can be invoked and additionally associated with two hidden properties: function context and function code. This might come as a surprise and unnatural, especially if you are coming from a more procedural programming background. Nevertheless this is the critical insight most of us need, to make sense of some of the strange ways that D3 uses function.

Note

JavaScript in its current form is generally considered not very object oriented, however, function object is probably one aspect where it outshines some of the other more object-oriented cousins.

Now with this insight in mind, let's take a look at the code snippet again:

  var instance = {}; // <-- A

  var headline, description; // <-- B

  instance.render = function () {
    var div = d3.select('body').append("div");

    div.append("h3").text(headline); // <-- C

    div.attr("class", "box")
      .attr("style", "color:" + spec.color) // <-- D
      .append("p")
      .text(description); // <-- E

    return instance; // <-- F
  };

At line marked as A, B, and C we can clearly see that instance, headline, and description are all internal private variables belonging to the SimpleWidget function object. While the render function is a function associated with the instance object which itself is defined as an object literal. Since functions are just an object it can also be stored in an object/function, other variables, arrays, and being passed as function arguments. The result of the execution of function SimpleWidget is the returning of object instance at line I.

function SimpleWidget(spec) {
...
  return instance; // <-- I
}

Note

The render function uses some of the D3 functions that we have not covered yet, but let's not pay too much attention to them for now since we will cover each of them in depth in the next couple of chapters. Also they basically just render the visual representation of this widget, not having much to do with our topic on hand.

Tip

Downloading the example code

You can download the example code files for all Packt books you have purchased from your account at http://www.packtpub.com. If you purchased this book elsewhere, you can visit http://www.packtpub.com/support and register to have the files e-mailed directly to you.

Static variable scoping

Curious readers are probably asking by now how the variable scoping is resolved in this example since the render function has seemingly strange access to not only the instance, headline, and description variables but also the spec variable that is passed into the base SimpleWidget function. This seemingly strange variable scoping is actually determined by a simple static scoping rule. This rule can be thought as the following: whenever searching for a variable reference, variable search will be first performed locally. When variable declaration is not found (as in the case of headline on line C) then the search continues to the parent object (in this case SimpleWidget function is its static parent and headline variable declaration is found at line B). If still not found, then this process will continue recursively to the next static parent so on and so forth till it reaches global variable definition, if still not found then a reference error will be generated for this variable. This scoping behavior is very different from variable resolution rules in some of the most popular languages such as Java and C#; it might take some time to get used to, however don't worry too much about it if you still find it confusing. With more practice and keeping static scoping rule in mind you will be comfortable with this kind of scoping in no time.

Tip

One word of caution here—again for folks from Java and C# backgrounds—is that JavaScript does not implement block scoping. The static scoping rule we described only applies to function/object but not at the block level.

for(var i = 0; i < 10; i++){
  for(var i = 0; i < 2; i++){
    console.log(i);
  }
}

Tip

You might be inclined to think this code should produce 20 numbers. However in JavaScript this code creates an infinite loop. This is because JavaScript does not implement block scoping so the i in the inner loop is the same i used by the the outer loop. Therefore it gets reset by the inner loop thus and can never end the outer loop.

This pattern is usually referred as functional when compared with the more popular prototype-based Pseudo-classical pattern. The advantage of the functional pattern is that it provides a much better mechanism for information hiding and encapsulation since the private variables—in our case the headline and description variables—are only accessible by nested functions via the static scoping rule therefore the object returned by the SimpleWidget function is flexible yet more tamper-proof and durable.

If we create an object in the functional style, and if all of the methods of the object make no use of this, then the object is durable. A durable object is simply a collection of functions that act as capabilities.
(Crockfort D. 2008)

Variable-parameter function

Something strange happens on line G:

instance.headline = function (h) {
  if (!arguments.length) h; // <-- G
  headline = h;
  return instance; // <-- H
};

You might be asking where this arguments variable on line G came from. It was never defined anywhere in this example. The arguments variable is a built-in hidden parameter that is available to functions when they are invoked. The arguments variable contains all arguments for a function invocation in an array.

Tip

In fact, arguments is not really a JavaScript array object. It has length and can be accessed using an index, however it does not have many of the methods associated with a typical JavaScript array object such as slice or concat. When you need to use a standard JavaScript array method on arguments, you need to use the apply invocation pattern:

var newArgs = Array.prototype.slice.apply(arguments);

This hidden parameter when combined with the ability to omit function argument in JavaScript allows you to write a function like instance.headline with unspecified number of parameters. In this case, we can either have one argument h or none. Because arguments.length returns 0 when no parameter is passed; therefore the headline function returns h if no parameter is passed, otherwise it turns into a setter if parameter h is provided. To clarify this explanation let's take a look at the following code snippet:

var widget = SimpleWidget({color: "#6495ed"})
    .headline("Simple Widget"); // set headline
console.log(widget.headline()); // prints "Simple Widget"

Here you can see how headline function can be used as both setter and getter with different parameters.

Function chaining

The next interesting aspect of this particular example is the capability of chaining functions to each other. This is also the predominant function invocation pattern that the D3 library deploys since most of the D3 functions are designed to be chainable to provide a more concise and contextual programming interface. This is actually quite simple once you understand the variable-parameter function concept. Since a variable-parameter function—such as the headline function—can serve as setter and getter at the same time, then returning the instance object when it is acting as a setter allows you to immediately invoke another function on the invocation result; hence the chaining.

Let's take a look at the following code:

var widget = SimpleWidget({color: "#6495ed"})
  .headline("Simple Widget")
  .description("This is ...")
  .render();

In this example, the SimpleWidget function returns the instance object (as on line I). Then, the headline function is invoked as a setter, which also returns the instance object (as on line H). The description function can then be invoked directly on return which again returns the instance object. Then finally the render function can be called.

Now with the knowledge of functional JavaScript and a working ready-to-go D3 data visualization development environment, we are ready to dive into the rich concepts and techniques that D3 has to offer. However before we take off, I would like to cover a few more important areas—how to find and share code and how to get help when you are stuck.

There's more...

Let's take a look at some additional helpful resources.

Finding and sharing code

One of the great things about D3 when compared with other visualization options is that it offers a wealth of examples and tutorials that you can draw your inspiration from. During the course of creating my own open source visualization charting library and the creation of this book, I had drawn heavily on these resources. I am going to list some of the most popular options available in this aspect. This list is by no means a comprehensive directory but rather a starting place for you to explore:

The D3 gallery (https://github.com/mbostock/d3/wiki/Gallery) contains some of the most interesting examples that you can find online regarding D3 usage. It contains examples on different visualization charts, specific techniques, and some interesting visualization implementations in the wild, among others.
BioVisualize(http://biovisualize.github.io/d3visualization) is another D3 gallery with categorization, to help you find your desired visualization example online quickly.
The D3 tutorials page (https://github.com/mbostock/d3/wiki/Tutorials) contains a collection of tutorials, talks and slides created by various contributors over time, to demonstrate in detail how to use a lot of D3 concepts and techniques.
D3 plugins (https://github.com/d3/d3-plugins). Maybe some features are missing in D3 for your visualization needs? Before you decide to implement your own, make sure to check out D3 plugin repository. It contains a wide variety of plugins that provide some of the common and, sometimes, uncommon features in the visualization world.
The D3 API (https://github.com/mbostock/d3/wiki/API-Reference) is very well documented. This is where you can find detailed explanations for every function and property that the D3 library has to offer.
Mike Bostok's Blocks (http://bl.ocks.org/mbostock) is a D3 example site, where some of the more intriguing visualization example can be found and which is maintained by its author Mike Bostock.
JS Bin (http://jsbin.com/ugacud/1/edit) is a pre-built D3 test and experiment environment completely hosted online. You can easily prototype a simple script using this tool or share your creation with other members in the community.
JS Fiddle (http://jsfiddle.net/qAHC2/) is similar to JS Bin; it also is a hosted-online JavaScript code prototyping and sharing platform.

How to get help

Even with all the examples, tutorial, and cookbook like this, you might still run into challenges when creating your visualization. Good news here is that D3 has a broad and active support community. Simply "googling" your question can most often yield a satisfying answer. Even if it does not, don't worry; D3 has a robust community-based support:

D3.js on Stack Overflow (http://stackoverflow.com/questions/tagged/d3.js): Stack Overflow is the most popular community-based free Q&A site for technologists. D3 is a specific category on the Stack Overflow site to help you reach the experts and get an answer to your question quickly.
The D3 Google group (https://groups.google.com/forum/?fromgroups#!forum/d3-js): This is the official user group for not just D3 but also other related libraries in its ecosystem.

Filter reviews by

All

Amazon verified reviews

gy21 Jan 08, 2016

Before you purchase the book, you should level yourself first. If you are a beginner of D3, then this book is right for you. It's starting with very basic concepts such as data binding, selectors, data manipulation, scale and etc. If you are expecting a comprehensive Angular/D3 book, this might not right for you.

Amazon Verified review

Mike Sloan Sep 10, 2015

As promised

Daniele Pagliero Jun 08, 2015

I found it very useful to move my steps in the data visualization field with d3. Even it's not a reference book, the recipes cover all the crucial aspects of the data viz workflow and can be easily changed to fit production needs.

Ewan Nicolson Apr 20, 2015

I've read a few other books on d3, this is the one that most clearly describes the underlying data model. I'm not sure how useful it would be to a complete beginner, but if you have some experience with Javascript already, or you've gone through the online tutorials, I think this is a good book to help you really understand what is going on inside d3.

Shmeets Jul 24, 2014

Doesn't really build beyond the basic examples on the d3 site, unfortunately.

Data Visualization with D3.js Cookbook: Turn your digital data into dynamic graphics with this exciting, leading-edge cookbook. Packed with recipes and practical guidance it will quickly make you a proficient user of the D3 JavaScript library.

What do you get with Print?

How to get source code

Tip

Setup a local HTTP server

Python Simple HTTP Server

Note

Node.js HTTP Server

Tip

Functions are objects

Static variable scoping

Variable-parameter function

Function chaining

Finding and sharing code

How to get help

Key benefits

Description

Who is this book for?

What you will learn

Product Details

What do you get with Print?

Product Details

Packt Subscriptions

Frequently bought together

Table of Contents

Recommendations for you

Customer reviews

Filter reviews by

People who bought this also bought

About the author

FAQs