[{"id":"ch01lvl1sec01","contentType":"text","title":"Introduction","content":"

Introduction

This chapter will introduce the D3.js (data-driven document) JavaScript library, describing its main features, explaining how it works, and showing how it drives data to transform documents. D3 contains an integrated set of tools that will help you bind data to graphical elements in a web page, in order to create scatterplots, bar charts, line charts, hierarchical node-link diagrams, networks, chord diagrams, sunburst charts, thematic geographic maps, or any interactive data visualization you can imagine. But D3.js is also a huge library, and is famous for having a steep learning curve. The goal of this book is to provide a learning path that will help you grasp its fundamental data-driven concepts and become familiar with its essential API.

In this chapter, you will learn how to set up your environment and test it by creating a very simple D3 application. It also includes a general overview of D3’s architecture, describing the relationships between its many modules and a brief description of each one.

This chapter will outline the following topics:

D3 data-driven documents
Using D3
Modules microlibraries

","url":"https://subscription.packtpub.com/book/web-development/9781838645571/1","metaDescription":"","titleSlug":"introduction"},{"id":"ch01lvl1sec02","contentType":"text","title":"D3 data-driven documents","content":"

D3 data-driven documents

D3, which stands for data-driven documents, is an open source JavaScript library used to create interactive web-based data visualizations. It provides a mechanism that connects arbitrary data to document elements, allowing their appearance and behavior to be driven by the data. Created by Mike Bostock, Jeff Heer, and Vadim Ogievetsky in 2001, it's currently used in hundreds of thousands of websites and is one of the most popular JavaScript data visualization libraries in the world.

If you have ever used interactive data applications from large news web portals such as The New York Times, The Washington Post, or The Guardian, there is a great probability that it was a D3 application. You may have also used one of the many charting libraries that are based on D3.

D3.js is also free and open source. You can use it in any project, commercial or not. Its source code is distributed in GitHub and is maintained by an active community of developers worldwide.

What is D3?

Yes, D3 is a JavaScript library, but no, D3 is not a charting library. There are no ready-to-use templates to create bar, pie, or line charts, for example. To create one of these charts, you have to draw all the lines, curves, and rectangles yourself using open standards such as SVG or HTML Canvas. D3, however, will do most of the hard work for you. It’s not trivial to use pure SVG to draw a bar chart; you need to scale data values so they fit in the chart, then calculate where to place each bar, and finally, set the coordinates of each rectangle before drawing it. Using D3, starting with an array of data, you can render all the bars with half a dozen chained commands in a single line of code.

D3 is a data visualization library. There are layout generators for pie charts that compute angles, which you can then use to draw arcs for the slices. There are functions that take a flat object array and turn it into a hierarchically linked object structure with coordinates for each node. You can use that data to draw circles at each coordinate point and draw lines between two nodes, rendering a tree. But you can also use the data differently, it's up to you. D3 doesn't restrict your creativity in any way. It doesn't tie you to a proprietary framework. Everything is based on open web standards.

D3 is also not only a data visualization library. Visualization is provided by HTML, CSS, or SVG. D3 focuses on the data. It's actually a collection of integrated JavaScript tools for manipulating the data structures necessary to create data visualizations. The core of the library is a fluent API used to select and manipulate the DOM. It replaces the DOM and libraries such as JQuery. It includes the data-driven mechanism that gives D3 its name, allowing you to bind arbitrary data to DOM elements, and then perform style and attribute transformations based on that data. This API is also used to bind and dispatch events, and to generate animated transitions.

D3 also includes tools to load and parse different data formats, such as JSON and CSV, perform general data manipulation on objects and arrays; generate data sequences and random numbers, perform interpolation and locale formatting. The actual data visualization parts contain layout generators, scales, axis generators, map projections, shape generators, color schemes, and other tools that are applied to previously selected DOM nodes and data.

How does it work?

A simplified view of D3's architecture is illustrated as follows. As implied by the name of the library, it's the data that drives the documents that display D3 visualizations. By adding, changing, and removing data, you directly affect the way your chart appears on the screen:

D3.js architecture

Data is usually provided as a JavaScript array, either generated locally or loaded from an external file. A typical D3.js script uses CSS selectors to select HTML or SVG elements and binds them to individual data items, removing, updating, or appending graphical elements automatically, when necessary.

You use CSS selectors to select one or more elements to create what D3 calls a selection object, which can then be used to apply styling and change the attributes of the selected elements. Binding an array to a selection object will map each data item to an element in the selection. These data items can be accessed in callback functions used in methods that set values for styles, attributes, and transforms.

You can also declare a selection of elements that don't exist in the DOM. Binding this selection to a data array will automatically create one element for each data item. You can then use the data to provide content, styles, and attributes for the new elements.

D3 also keeps data and their elements in sync. When updating with a smaller dataset, elements in excess are placed in a separate array, so you can remove them. If the dataset grows, existing elements are updated and missing elements are created and appended to fit the available data.

","url":"https://subscription.packtpub.com/book/web-development/9781838645571/1/ch01lvl1sec02/d3-data-driven-documents","metaDescription":"","titleSlug":"d3-data-driven-documents"},{"id":"ch01lvl1sec03","contentType":"text","title":"Using D3","content":"

Using D3

All that you need to start using D3 can be found at d3js.org where you can download and install the library as a single JavaScript file, a collection of standalone microlibraries, a CDN link, or an NPM installation script. The site also includes the official documentation, which covers the library in great detail, with hundreds of tutorials for beginners and advanced users, and thousands of online examples.

The D3 site is a great place to start because you can try out several examples of data visualizations created with D3. Just click on any hexagon and it will lead you to a page showing a D3 application and its source code. Most are published in GitHub and published using the Bl.ocks platform (bl.ocks.org), which is a huge portfolio based on GitHub source code. Newer projects use the Observable platform (observablehq.com), which contains interactive tutorials where you can edit the code and see changes in real time. These platforms are very popular among the D3 community. Many of these tutorials were designed by creators and major contributors of the D3 library, such as Mike Bostock, Jason Davies, and Philippe Riviere.

Take some time to explore these examples and see what you can create using D3.

The D3.js website is a showcase of the many different data visualizations you can create using this library

Environment setup

You don't need a sophisticated development environment to use D3. If you already develop Web applications using npm, you can install it with:

npm install d3

You can also add the d3.js file to a local folder after downloading the full library (default bundle) as a ZIP from the official website. Then you can import it to your HTML file using the <script> tag and a local relative path; for example (for the minified version):

<script src="js/d3/d3.v5.min.js"></script>

If you have a permanent web connection, it's probably simpler to use a CDN link:

<script src="https://d3js.org/d3.v5.min.js"></script>

For very simple applications that don't load external files, you can simply open the page in your browser from the file system. It's better to load the page using a web server. If you are using a code editor, it may already have a launch configuration or plugin that always starts a web server when you choose to load a page. You can also install a simple web server using npm, by using the following command:

npm install http-server –g

The preceding command will install it globally. Then, you can move to the directory where your files are located and simply type the following:

http-server

Then you can load your files from http://localhost:8080.

You can also develop D3 using online code editors, such as CodePen (codepen.io) or JSFiddle (jsfiddle.net). It’s also a great way to share your code.

Hello, world

Let's create a simple D3-powered data-driven visualization to test your configuration. Create a new HTML file in your development environment and import the D3 library, or import it into any HTML page using the <script> tag as shown in the last section. Then add an <svg> and a <script> block inside your page's <body> as follows:

<body>
   <svg id="chart" width="600" height="200"></svg>
    <script> 
    </script>
</body>

Now add the following array inside the <script> block:

const array = [100, 200, 300, 350, 375, 400, 500];

We will use that array to generate a series of dots. Type in the following code (you can ignore the comments), which consists of a series of chained commands:

d3.select("#chart")     // selects the svg element by id (like JQuery)
  .selectAll("circle")  // declares the elements we would like to create
  .data([100])          // sets the data to drive the creation of the 
                        // elements
  .enter()              // creates a selection to add elements per 
                        // data item
  .append("circle")     // appends an element of this type to each 
                        // data item
    .attr("r", 10)      // sets “r” attribute
    .attr("cy", 100)    // sets “cy” attribute
    .attr("cx", d => d) // sets “cx” attribute (same as function(d) { 
                        // return d; }

If your configuration is working correctly, when you load the page containing this code in a browser, you should see a single dot on the screen:

A dot on a screen created with D3. Code: HelloWorld/1-intro-d3.html

If you inspect the dot with your browser's JavaScript development tools, you will notice that the following code was generated inside the SVG:

<svg width="600" height="200">
    <circle r="10" cy="100" cx="100"></circle>
</svg>

The preceding JavaScript code selects the <svg> element from the page and then selects all <circle> elements inside it. But there aren’t any, so the selection is empty. The next line, however, contains a data() command with a one-element array: [100]. The enter() command binds the data to the selection creating a selection of <circle> placeholders the same size as the data array. In this case, it will only contain one placeholder. Finally, the append() command, called in the context of this selection, appends a <circle> element to it, making it a child of the <svg> element.

The last three commands set attribute values and the last one takes a function and returns the element received by the function, which is used as the attribute's value. This element is the value 100 that was stored in the array passed to the data() command.

The code is available in the HelloWorld/1-intro-d3.html file from the GitHub repository for this chapter.

Now let's make some changes. Replace the [100] in the data() command with array. It will now reference the seven-element array we created before:

.data(array)

Run the page again. What happened? Consider the following screenshot:

Several dots added to the screen with positions driven by the data. Code: HelloWorld/2-binding.html

Now there are seven dots on the screen; the same number of dots as the data array. And each dot is positioned at a different horizontal position (which was defined by the cx property of the <circle>) that is set using values from the data array. We used the data array to drive the position of each circle.

Now skip a line and add this code after the selection (see HelloWorld/3-update.html):

setTimeout(function() {
    d3.select("#chart").selectAll("circle")
      .data([50, 75, 125, 225, 325, 425, 450])
        .attr("r", 5)
        .attr("cx", d => d)
        .style("fill", "red")
}, 2000)

This code will run two seconds after the page loads and shows the circles in the positions defined by the first chain of commands. The function inside it selects the #chart SVG element again, and then all the circles inside it. But this time, these circles do exist. The data() command binds a new data array to the selection. No enter() command is called because there aren't any elements to be added.

The attr() commands are used to the circle's attributes. Only two attr()commands were called in the preceding code because we are only changing two attributes: the radius of the circle, and the position, which will now obtain its value from the new array. Besides that, we also used style() to change the color of the dots. If you run this file, two seconds later, the dots shrink, move to the left and become red:

The dots changed color, size, and position after an update. Code: HelloWorld/3-update.html

One very nice feature of D3 is how simple it is to animate transitions. You just need to chain a transition() command before the attributes and styles that changed. The default transition takes a quarter of a second. Let's make it a little longer by configuring duration. Add the following line between the data() command and the first attr(), to add a one-second transition during the data update:

.transition().duration(1000)

Your page body should now look like this (see HelloWorld/4-transition.html):

<body>
<svg id="chart" width="600" height="200"></svg>
<script>
    const array = [100, 200, 300, 350, 375, 400, 500];

    d3.select("#chart")
            .selectAll("circle")
            .data(array)
            .enter()
            .append("circle")
            .attr("r", "10")
            .attr("cy", 100)
            .attr("cx", d => d)

    setTimeout(function() {
        d3.select("#chart").selectAll("circle")
          .data([50, 75, 125, 225, 325, 425, 450])
          .transition().duration(1000)
            .attr("r", 5)
            .attr("cx", d => d)
            .style("fill", "red")
    }, 2000)
</script>
</body>

Now, after two seconds, the dots will spend another second changing color, shrinking, and moving to the left. Congratulations! You created a full D3 application, complete with data updates and transitions.

Debugging D3

Although you don't need a full frontend modular development environment to create visualizations with D3.js, you still need a good debugger. Every browser comes with development tools that allow you to navigate a static page structure and generated DOM elements, and a console where you can interact in real time with the data used by the JavaScript engine in the real time.

The most important tool is the JavaScript console, where you will see any error messages. It's very common to get a blank page when you expected something else and to not have a clue on why your code doesn't work as expected. Sometimes it's just a comma you forgot, or the internet is down and some file was not loaded. If you have the JavaScript console open while you run your page, it will instantly tell you what's going on. It's also a good idea to use an editor with line numbering since most error messages inform the lines where the problem occurred:

Debugging JavaScript with the JavaScript console

You can open the developer tools as a frame in your browser or as a separate window. Following are the menu paths for the JavaScript console in latest versions of the three most popular browsers:

Chrome: View | Developer | JavaScript Console
Firefox: Tools | Web Developer | Web Console
Safari: Develop | Show Error Console

Many code fragments and examples can be tested by simply typing them in the JavaScript console. The console’s context is the currently loaded page. You will be able to access the functions of any JavaScript library file that was loaded with the <script> tag, and any global variables declared in <script></script> blocks, so you can also use the console to experiment with D3. The console is a great way to learn D3 since you can run functions in real time and immediately see the results they produce. You can try out many code examples in this book using the JavaScript console.

","url":"https://subscription.packtpub.com/book/web-development/9781838645571/1/ch01lvl1sec03/using-d3","metaDescription":"","titleSlug":"using-d3"},{"id":"ch01lvl1sec04","contentType":"text","title":"Modules (microlibraries)","content":"

Modules (microlibraries)

You don't have to always load the entire D3 library. D3 is a modular library, so if you are only using some D3 features, you can include only the parts that you are using. The minified default bundle is about 240 KB in size, but you may be able to create your chart using as little as 13 KB if you don't need many modules. An animated interactive bar chart, complete with tooltips, transitions, colors, and SVG graphics can be created with less than 24 KB loading just the following two modules:

<script src="https://d3js.org/d3-selection.v1.min.js"></script>
<script src="https://d3js.org/d3-transition.v1.min.js"></script>

But if you need axes, maps, and other features, you will require more modules and dependencies. In this case, either use the default bundle, or set up a development environment where you can install each module using npm, since it automatically includes any dependencies. For production, you can generate and export a custom bundle using a packing tool such as Rollup. To install any module using npm, use the following:

npm install module-name

For the examples in this chapter (and most of the book) we will import D3 using the <script> tag with the CDN URL to the default bundle.

Even if you always use the default d3.js bundle, you should be aware of the modules it contains, and of the modules that are not part of the default bundle, because they need to be imported separately in case you require their functions. All the official documentation is also organized per module, and knowledge of the modular structure will allow you to tune the performance of your D3 application, in case you decide to create a custom bundle. Modules have their own versioning systems. You might need to load a module separately if you wish to use a feature included in a new major version, not yet included in the default bundle.

The following diagram shows the modules available in D3.js version 5, indicating direct dependencies (transitive dependencies are not shown). For example, if you need to use a function from the d3-scale module, you should import all the direct dependencies: (d3-array, d3-time-format, d3-collection, d3-format, d3-interpolate) and the transitive ones:( d3-time and d3-color). In this case, you should either use npm or import the default bundle.

Modules (microlibraries) available in D3.js v5 showing direct dependencies. The orange and yellow libraries are included as part of the default bundle. The other libraries (pink and purple) have to be explicitly imported

The following tables contain a quick reference of all modules available in the current version, classified according to their purpose. In this book, we will be using functions from almost all of them.

Data manipulation

The following modules listed are used to generate, manipulate, transform, parse, and format data, in the form of numbers, text, arrays, objects, and files. They are all included in the default d3.js bundle:

Module	Bundled (d3v5)	Description
d3-array	Yes	Several array utilities that extend the basic ES6 functions, optimized for use with datasets. Dependencies: none.
d3-collection	Yes	Maps and sets optimized for use with datasets; functions for object collections and nesting data. Dependencies: none.
d3-random	Yes	Random number generators. Dependencies: none.
d3-dsv	Yes	Parser functions for delimiter-separated data. Dependencies: none.
d3-interpolate	Yes	Several functions for interpolating numbers, colors, strings, and so on. Dependencies: d3-color.
d3-scale	Yes	Generator functions to map data dimensions to graphical dimensions. Dependencies: d3-array, d3-collection, d3-format, d3-interpolate, d3-time-format, d3-time.
d3-time	Yes	API for operations with time (intervals, ranges, and so on). Dependencies: none.
d3-format	Yes	Locale-sensitive methods for number formatting. Dependencies: none.
d3-time-format	Yes	Locale-sensitive methods for date/time formatting. Dependencies: d3-time.

Modules with methods for manipulating, transforming, parsing, and formatting data

Document manipulation

These are core modules in D3 used to select and manipulate HTML or SVG elements by providing a concise API to the DOM. With these modules, you can select and filter elements (using CSS selectors), create elements, append, insert, or remove from the DOM tree, add attributes and contents, change styles or classes, connect event handlers, and join data. Practically any D3 application uses at least d3-selection.

Module	Bundled (d3v5)	Description
d3-selection	Yes	Contains the essential DOM API for selection and manipulation of DOM elements. Dependencies: none.
d3-selection-multi	No	Adds optional support for setting multiple attributes, styles, or properties in selections and transitions using an object syntax. Dependencies: d3-selection, d3-transition.

Modules with functions for selecting and manipulating graphical elements using the DOM

Interactivity and animation

The modules listed in the following table are used in dynamic visualizations when updating data, zooming, dragging, selecting and clicking charts and maps.

Module	Bundled (d3v5)	Description
d3-transition	Yes	Methods to configure the transition applied to a selection. Dependencies: d3-interpolate, d3-ease, d3-dispatch, d3-selection, d3-timer, d3-color.
d3-ease	Yes	Easing functions for animations. Dependencies: none.
d3-zoom	Yes	Apply transforms in HTML, SVG, or Canvas using mouse or touch. Also includes support for programmatic transforms. Dependencies: d3-dispatch, d3-drag, d3-interpolate, d3-selection, d3-transition.
d3-drag	Yes	Drag and drop SVG, HTML, or Canvas using mouse or touch. Dependencies: d3-dispatch, d3-selection.
d3-brush	Yes	Selects a one or two-dimensional region for detailing using the mouse or touch. Dependencies: d3-dispatch, d3-drag, d3-interpolate, d3-selection, d3-transition.
d3-quadtree	Yes	Partitions two-dimensional space recursively into squares. Used to calculate regions for brushing and zooming. Dependencies: none.
d3-timer	Yes	A queue for managing concurrent animations. Dependencies: none.
d3-dispatch	Yes	Mechanism for event dispatching to named callbacks. Dependencies: none.

Modules with methods for event handling and dispatching, animations, and interactions

Colors

The following table lists modules that contain representations for machine-friendly and human-friendly color spaces and color schemes.

Module	Bundled (d3v5)	Description
d3-color	Yes	Support for several color spaces. Supports RGB, HSL, Cubehelix, Lab (CIELAB), and HCL (CIELCH). Dependencies: none.
d3-scale-chromatic	Yes	Sequential, diverging, and categorical color schemes and palettes. Dependencies: d3-color, d3-interpolate.
d3-hsv	No	Support for the HSV color space. Dependencies: d3-color.
d3-hcg	No	Support for the HCG color space. Dependencies: d3-color.
d3-cam16	No	Support for the CIECAM16 color space. Dependencies: d3-color.

Modules with methods for operations with colors spaces and schemes

Asynchronous operations and packaging

These modules are used to load files using Ajax.

Module	Bundled (d3v5)	Description
d3-fetch	Yes	Methods for fetching files asynchronously using Ajax. Also includes parsing methods for several data formats. Dependencies: d3-dsv.
d3-queue	No	Supports queueing of concurrent asynchronous requests. Dependencies: none. This module is deprecated in favor of ES6 promises.
d3-require	No	Supports AMD for loading modules and dependencies. Dependencies: none.

Modules for Ajax operations and module packaging

2D geometry

The following modules listed contain a complete two-dimensional graphical API that can be used to create any type of visualization, from simple line, area, or pie charts to arbitrary paths and shapes.

Module	Bundled (d3v5)	Description
d3-shape	Yes	Generators and utility functions to create and manipulate arcs, curves, lines, areas, stacks, curves, pie charts, symbols, and links in SVG or Canvas. Dependencies: d3-path.
d3-axis	Yes	Generates SVG axes. Dependencies: none.
d3-path	Yes	A Canvas API that generates SVG paths (which can be rendered as a Canvas or SVG). Dependencies: none.
d3-polygon	Yes	Utility functions for two-dimensional polygons. Dependencies: none.

Modules with methods for primitive geometric SVG operations and generators for shape-based charts

Spherical geometry and geographic maps

The following table lists modules that are used to display information in geographic maps. They contain methods for spherical trigonometry, geographic projections, and other utilities.

Module	Bundled (d3v5)	Description
d3-geo	Yes	Generators for geographic paths and standard projections, spherical shapes, and trigonometric functions, transforms, streams, and other utilities. Dependencies: d3-array.
d3-contour	Yes	Creates contour polygons, which are commonly used for geographical density, bathymetry, or relief maps. Dependencies: d3-array.
d3-geo-projection	No	Additional projections for d3-geo. Dependencies: d3-geo, d3-array.
d3-geo-polygon	No	Several utility functions for spherical polygons. Dependencies: d3-geo, d3-array, d3-geo-projection.

Modules with methods for displaying data from geographical information systems (GIS)

Layouts

The following modules listed, include generator functions, utilities, and algorithms to create visualizations for complex relationships such as node-link hierarchies, graphs, trees, networks, flow diagrams, tiles, and Voronoi diagrams.

Module	Bundled (d3v5)	Description
d3-hierarchy	Yes	Generator functions for hierarchic layouts such as node-link, adjacency, and enclosure diagrams. Functions create stratifications, clusters, trees, dendrograms, treemaps, circle packs, and partitions. Dependencies: none.
d3-force	Yes	Implementation of a force simulation (uses the Stormer-Verlet method), used for interactive visualizations of graphs, networks, and hierarchies. Dependencies: d3-collection, d3-dispatch, d3-quadtree, d3-timer.
d3-chord	Yes	Generator function and utilities for creating chord flow diagrams. Dependencies: d3-array, d3-path.
d3-sankey	No	Generator function and utilities for creating Sankey flow diagrams. Dependencies: d3-array, d3-collection, d3-shape.
d3-tile	No	A layout for raster image-based tiles, normally used for displaying tiles of geographic maps behind a vector layer. Dependencies: d3-array.
d3-hexbin	No	Generator function and utilities for creating scatterplots of hexagonal bins, useful for color-encoded heatmaps. Dependencies: none.
d3-voronoi	Yes	Generator functions and utilities for creating Voronoi diagrams (nature-like diagrams that show regions in a plane that are nearest to a point). Dependencies: none.
d3-delaunay	No	Compute the Delaunay triangulation of a set of points; a simpler and faster method to create Voronoi diagrams. Dependencies: none.

Modules containing algorithms and generator functions for graphical layouts

","url":"https://subscription.packtpub.com/book/web-development/9781838645571/1/ch01lvl1sec04/modules-microlibraries","metaDescription":"","titleSlug":"modules-microlibraries"},{"id":"ch01lvl1sec05","contentType":"text","title":"Summary","content":"

Summary

This chapter provided a general introduction to the D3.js library, describing its architecture, modules, and showing how to set up a small D3 application that, although very simple, demonstrated one of the central paradigms in D3, which is driving the appearance of a visualization using arbitrary data.

The next chapter will consist of reference topics that cover fundamental technologies used by D3, such as SVG (an introductory tutorial), JavaScript (mostly a review of the fundamental data structures in ES 2015), and Canvas (a short reference). There is also a short section on data formats. If you are comfortable with all these topics, or if you want to start using D3 right away, you can skip them now and proceed to straight Chapter 3, Quick Start.

","url":"https://subscription.packtpub.com/book/web-development/9781838645571/1/ch01lvl1sec05/summary","metaDescription":"","titleSlug":"summary"},{"id":"ch01lvl1sec06","contentType":"text","title":"References","content":"

References

D3.js documentation: d3js.org
D3.js API reference: github.com/d3/d3/blob/master/API.md
D3.js module repository: github.com/d3
D3.js gallery: github.com/d3/d3/wiki/Gallery
D3.js wiki: github.com/d3/d3/wiki
Bl.ocks portfolios: bl.ocks.org
Observable notebooks: observablehq.com

","url":"https://subscription.packtpub.com/book/web-development/9781838645571/1/ch01lvl1sec06/references","metaDescription":"","titleSlug":"references"}]

Method	Description
`angle(`function`)`	This...

Learn D3.js

By : Helder da Rocha

Learn D3.js

By: Helder da Rocha

Overview of this book

Related Content you might be interested in

Current Title:

Learn D3.js

Learn Chart.js

Data Visualization with D3 4.x Cookbook

D3.js 4.x Data Visualization