Book Image

Instant jQuery Flot Visual Data Analysis

By : Brian Peiris
Book Image

Instant jQuery Flot Visual Data Analysis

By: Brian Peiris

Overview of this book

Data visualization and analysis is a crucial skill in many software projects. Flot uses jQuery and HTML5 to add easy and powerful visualization capabilities to any web application. Flot produces beautiful visualizations with a minimal amount of code. It is also highly configurable, extensible, and includes many plugins out of the box. A practical guide to take you through the basics of using Flot to visualize your data; the book describes Flot's functionality in dedicated sections that include code, screenshots, and detailed explanations so that you can learn to produce charts in minutes. As you progress though this book, it will guide you through real-world examples to show you how to use statistical techniques with Flot for interactive data visualization and analysis. Starting with the very basics, you will learn exactly what you need to do to display the simplest chart using Flot. This step-by-step guide takes you through Flot's many features and settings until you can finally master the techniques you'll need to apply Flot to your application's data. You'll learn to create basic point, line, and bar charts and to use Flot's stack, pie, and time plugins to create specialized chart types. Along with learning to display complex data with multiple customizable axes you will learn to make your charts interactive with Flot's crosshair plugin. Finally, this book will guide you through learning statistical techniques via the jStat JavaScript library to analyse data; along with Flot's errorbars and fillbetween plugins to display error margins and data percentiles. Instant jQuery Flot Visual Data Analysis will give you a head start so that you can add data visualization features to your applications with ease.
Table of Contents (7 chapters)

Applying Flot (Should know)


Now that we've learned how Flot works, we can apply it to our own data sources. Since we can't cover all possible uses of Flot's capabilities, we'll try to demonstrate some examples of how Flot can be applied to real-world data.

There are a myriad of data sources out there and many of them offer their data freely as Open Data sources. One of those sources is The World Bank. They provide data on global statistics and indicators through a simple API.

How to do it…

We use jQuery's getJSON function to query The World Bank's Cellular Indicator API and manipulate the data before displaying pie charts with Flot:

    var getDataPage = function (url, cb, page, data) {
      return function (response) {
        var newData = response[1];
        newData.unshift(newData.length);
        newData.unshift(data.length);

        // append new data to existing data
        data.splice.apply(data, newData);

        // get more data if there are more pages
        if (response[0].pages > page) {
          getData(url, cb, page + 1, data);
        }
        else {
          cb(data);
        }
      };
    };

    var getData = function (url, cb, page, data) {
      page = page || 1;
      data = data || [];

      $.getJSON(
        url + '&page=' + page, 
        getDataPage(url, cb, page, data)
      );
    };

    var excludeInvalidData = function (d) {
        // Some data has a null value or actually 
        // represents groups of countries.
        return (
          d.value &&
          !/\d/.test(d.country.id) &&
          !/^x/i.test(d.country.id) &&
          ['OE', 'ZJ', 'ZQ', 'ZG', 'ZF', 'EU'].
            indexOf(d.country.id) === -1
        );
    };
    var makeFormatter = function (power, suffix) {
      return function (label, series) {
        return (
          label + '<br />' +
          (
            series.data[0][1] / Math.pow(10, power)
          ).toFixed(0) + ' ' +
          suffix
        );
      };
    };

    var options = {
      series: {
        pie: {
          show: true,
          label: {
            show: true,
            formatter: makeFormatter(6, 'M')
          }
        }
      }
    };

    var plotData = function (rawData) {
      var data = rawData.
        filter(excludeInvalidData).
        map(function (d) {
          return { 
            label: d.country.value, 
            data: parseFloat(d.value) 
          };
        });

      data.sort(function (a, b) { return b.data - a.data; });

      var topFive = data.slice(0, 5);
      $('#topFive').plot( topFive, options );

      var bottomFive = data.slice(-5);
      options.series.pie.label.formatter = 
        makeFormatter(3, 'K');
      $('#bottomFive').plot( bottomFive, options );
    };

    getData(
      'http://api.worldbank.org/' + 
      'countries/all/indicators/IT.CEL.SETS/?' +
      'format=jsonp&prefix=?&date=2011:2011',
      plotData
    );

The resulting pie charts depict the number of cellular subscription for the top five and bottom five countries in the world:

How it works…

The API returns a JSON structure that includes several pieces of information, but we are really only interested in the country name and the number of cellular subscriptions. We first filter out some invalid data with JavaScript's filter function, and then map the relevant values to the format that Flot expects for pie charts.

We sort the data, and then extract the top and bottom five countries. Finally, we display pie charts for the data sets with a custom pie label formatter that shows the number of subscriptions in millions or thousands.

There's more…

The following example shows some personal data analysis. We take weight loss data collected over several years and analyze it using custom averaging and trend calculations:

...
  <script>
    var weightData;
    var goalWeight = 70;
    var MS_PER_DAY = 24 * 60 * 60 * 1000;

    var parseDate = function (d) {
      // Turn a string like '2013-08-18'
      // into a timestamp.
      var dateParts = d[0].split('-');
      var timestamp = new Date(
        parseInt(dateParts[0], 10),
        parseInt(dateParts[1], 10) - 1,
        parseInt(dateParts[2], 10)
      ).getTime()
      return [ timestamp, d[1] ];
    };

    var calculateRunningAvg = function (data, i, arr) {
      // return a data point which has a y-value
      // that is the average of the previous 10 y-values
      // in the array.
      var pointsToAverage = 
        arr.slice(Math.max(0, i - 9), i + 1);
      var sum = pointsToAverage.reduce(function (a, b) {
          return a + b[1]; 
      }, 0);
      var average =  sum / Math.min(i + 1, 10);
      return [ data[0], average ];
    };

    var calculateSlope = function (p1, p2) {
      var rise = p2[1] - p1[1]
      var run = p2[0] - p1[0];
      return rise / run;
    };

    var calculateTrend = function (arr) {
      var n = arr.length - 1;
      var slope = calculateSlope(arr[n - 1], arr[n]);
      var trend = [];
      var start = arr[n];

      // If the slope is positive or close to 0, 
      // calculate a trend 10 days in to the future.
      // Otherwise, calculate the trend until it
      // meets the goal weight.
      for (
        var i = 0, y = start[1];
        slope >= -1e-10 ? i < 10 : y > goalWeight;
        i++
      ) {
        var x = start[0] + i * MS_PER_DAY;
        y = start[1] + i * MS_PER_DAY * slope;
        trend.push([x, y]);
      }
      return trend;
    };

    var filterToPeriod = function (data, period) {
      if (period) {
        var start = data[data.length - 1][0] - period;
        data = data.filter(function (d) {
          return d[0] > start; 
        });
      }
      return data;
    };
    var plotWeight = function (rawData, period) {
      weightData = weightData || rawData;

      var data = rawData.map(parseDate);

      data = filterToPeriod(data, period);

      var runningAverage = data.map(calculateRunningAvg);

      var trend = calculateTrend(runningAverage);

      var goalWeightLine = [
        [data[0][0], goalWeight],
        [trend[trend.length - 1][0], goalWeight]
      ];

      $('#weight').plot(
        [
          data,
          { label: 'Average', data: runningAverage },
          { label: 'Trend', data: trend },
          { label: 'Goal weight', data: goalWeightLine },
        ],
        {
          xaxis: { mode: 'time' },
          legend: { show: true, position: 'sw' }
        }
      );
    };

    $('#oneYear').click(function () {
      plotWeight(weightData, 12 * 30 * MS_PER_DAY);
    });
    $('#threeMonths').click(function () {
      plotWeight(weightData, 3 * 30 * MS_PER_DAY);
    });
    $('#oneMonth').click(function () {
      plotWeight(weightData, 30 * MS_PER_DAY);
    });
    $('#all').click(function () {
      plotWeight(weightData, null);
    });

    plotWeight([
      ['2010-01-20',97.25],
      ['2010-01-21',96.98],
      ['2010-01-22',97.25],
      ...
      ['2013-08-29',77.30],
      ['2013-08-30',75.00],
      ['2013-09-02',74.60]
    ]);

  </script>
...

The resulting chart shows the raw weight data with an overlaid running average, a trend line and a goal line. The code also adds interactivity to the chart via buttons that dynamically change the visible date range; effectively allowing the user to zoom in on the data.