Book Image

Building Data-Driven Applications with Danfo.js

By : Rising Odegua, Stephen Oni
Book Image

Building Data-Driven Applications with Danfo.js

By: Rising Odegua, Stephen Oni

Overview of this book

Most data analysts use Python and pandas for data processing for the convenience and performance these libraries provide. However, JavaScript developers have always wanted to use machine learning in the browser as well. This book focuses on how Danfo.js brings data processing, analysis, and ML tools to JavaScript developers and how to make the most of this library to build data-driven applications. Starting with an overview of modern JavaScript, you’ll cover data analysis and transformation with Danfo.js and Dnotebook. The book then shows you how to load different datasets, combine and analyze them by performing operations such as handling missing values and string manipulations. You’ll also get to grips with data plotting, visualization, aggregation, and group operations by combining Danfo.js with Plotly. As you advance, you’ll create a no-code data analysis and handling system and create-react-app, react-table, react-chart, Draggable.js, and tailwindcss, and understand how to use TensorFlow.js and Danfo.js to build a recommendation system. Finally, you’ll build a Twitter analytics dashboard powered by Danfo.js, Next.js, node-nlp, and Twit.js. By the end of this app development book, you’ll be able to build and embed data analytics, visualization, and ML capabilities into any JavaScript app in server-side Node.js or the browser.
Table of Contents (18 chapters)
1
Section 1: The Basics
3
Section 2: Data Analysis and Manipulation with Danfo.js and Dnotebook
10
Section 3: Building Data-Driven Applications

Setting up a modern JavaScript environment with transpilers

One of the unique features of JavaScript, and the reason why it is very popular, is its cross-platform support. JavaScript runs almost everywhere, from browsers and desktops to even on the server side. While this is a unique feature, getting JavaScript to run optimally in these environments requires some setup and configuration using third-party tools/libraries. Another reason why you need to set up tooling is that you can write JavaScript in different flavors, and because these modern/newer flavors may not be supported by older browsers. This means that the code you write in newer syntax, typically post-ES15, will need to be transpiled into pre-ES16 format for it to run properly in most browsers.

In this section, you will learn how to set up and configure a JavaScript project to support cross-platform and modern JavaScript code. You will use two popular tools – Babel and webpack – to achieve this.

Babel

Babel is a tool for converting JavaScript code written in ES15 code into a backward-compatible version of JavaScript in modern or older browsers. Babel can help you to do the following:

  • Transform/transpile syntax.
  • Polyfill features that are missing in your target environment. Some modern features that are not available in older environments are automatically added by Babel.
  • Transform source code.

In the following code, we show an example of a Babel-transformed code snippet:

// Babel Input: ES2015 arrow function
["Lion", "Tiger", "Shark"].map((animal) => console.log(animal));
 
// Babel Output: ES5 equivalent
["Lion", "Tiger", "Shark"].map(function(animal) {
  console.log(animal)
});

You will notice that in the preceding code snippet, the modern arrow function is automatically transpiled to the function keyword that is supported by all browsers. This is what Babel does under the hood to your source code.

Next, let's understand where webpack comes in.

Webpack

webpack is also a transpiler, and can perform the same function as Babel, and even more. webpack can package and bundle just about anything, including images, HTML, CSS, and JavaScript, into a single optimized script that can easily be used in the browser.

In this section, we'll leverage both Babel and webpack to show you how to set up a cross-platform JavaScript project. Let's dive right in.

Example project using Babel and webpack

In this section, we're going to create a simple JavaScript project using npm. As such, you should have Node.js installed locally in order to follow along. Perform the following steps to achieve this:

  1. Open a terminal in your preferred directory and create a folder with the following commands:
    $ mkdir cross-env-js
    $ cd cross-env-js

    This will create a folder, cross-env-js, in your directory, and then change the directory as well.

  2. Create a package.json file. While you can do this manually, it is easier to create one using npm. Run the following command in your terminal:
    $ npm init -y

    The preceding code will create a package.json file and accept all default options. Ideally, this should output the following:

    Figure 1.1 – Output from running the npm init –y command

    Figure 1.1 – Output from running the npm init –y command

  3. Next, install all the relevant packages that will help us to perform bundling and transpilation:
    $ npm install --save-dev @babel/core @babel/cli @babel/preset-env babel-loader webpack webpack-cli 
    $ npm install --save @babel/polyfill

    Note that you install most packages as development dependencies. This is important so that your final code is not bundled with packages you only need during development. After installing these packages, your package.json file should look like this:

    {
      "name": "cross-env-js",
      "version": "1.0.0",
      "description": "",
      "main": "index.js",
      "scripts": {
        "test": "echo \"Error: no test specified\" && exit 1"
      },
      "keywords": [],
      "author": "",
      "license": "ISC",
      "devDependencies": {
        "@babel/cli": "^7.12.8",
        "@babel/core": "^7.12.9",
        "@babel/preset-env": "^7.12.7",
        "babel-loader": "^8.2.2",
        "webpack": "^5.9.0",
        "webpack-cli": "^4.2.0"
      },
      "dependencies": {
        "@babel/polyfill": "^7.12.1"
      }
    }
  4. Add some code, which we'll transpile and test. For this section, you can either create files and folders from the terminal or use a code editor. I'll use the Visual Studio Code editor here.

    In your code editor, open the cross-env-js project folder and then create the files and folders as follows:

    ├── dist
    │   └── index.html
    ├── src
    │   ├── index.js
    │   ├── utils.js

    That is, you will create two folders called dist and src. dist will contain an HTML file (index.html), which will be used to test our bundled application, and src will contain our modern JavaScript code that we want to transpile.

    After creating these files and folders, your entire directory structure should look like this:

    ├── dist
    │   └── index.html
    ├── node_modules
    ├── package-lock.json
    ├── package.json
    └── src
        ├── index.js
        └── utils.js

    Note

    If you're using version control such as Git, you will typically add a .gitignore file to specify that node_modules can be ignored.

  5. Create a dist folder, and in that folder, create an index.html file with the following code:
    <!DOCTYPE html>
    <html lang="en">
    <head>
        <meta charset="UTF-8">
        <meta name="viewport" content="width=device-width, initial-scale=1.0">
        <script src="bundle.js"></script>
        <title>Cross Environment Support</title>
    </head>
    <body>
        
    </body>
    </html>

    The HTML file should be familiar to you, but notice that we added a script tag pointing to a bundle.js file. This file does not yet exist and will be generated by webpack using Babel under the hood.

  6. Write some modern JavaScript in the src folder. Starting with utils.js, we'll create and export some functions, and then import them to be used in index.js.

    Starting with utils.js, add the following code:

    const subjects = {
        John: "English Language",
        Mabel: "Mathematics",
        Mary: "History",
        Joe: "Geography"
    }
     
    export const names = ["John", "Mabel", "Mary", "Joe"]
    export const getSubject = (name) =>{
        return subjects[name]
    }

    The utils.js script uses some modern JS syntax, such as export and arrow functions, and these will only be compatible with older browsers after transpilation.

    Next, in the index.js script, you'll import these functions and use them. Add the following code to your index.js script:

    import { names, getSubject } from "./utils";
    names.forEach((name) =>{
        console.log(`Teacher Name: ${name}, Teacher Subject: ${getSubject(name)}`)
    })

    You'll notice that we are also using arrow functions and the destructuring import in the index.js file. Here, we're importing the exported array (names) and the getSubject function from the utils.js script. We are also using a combination of the arrow function and template literals (` `) to retrieve and log the details of each Teacher.

  7. Now that our modern JS files are ready, we'll create a configuration file that tells webpack where to find our source code to bundle as well as which transpiler to use, in our case, Babel.

    In your root directory, create a webpack.config.js file and add the following code:

    const path = require('path');
    module.exports = {
      entry: './src/index.js',
      output: {
        filename: 'bundle.js',
        path: path.resolve(__dirname, 'dist'),
        publicPath: '/dist'
      },
      module: {
        rules: [
          {
            test: /\.js$/,
            exclude: /(node_modules)/,
            use: {
              loader: 'babel-loader',
            }
          }
        ]
      }
    };

    Let's understand what is going on in this file:

    a) The first part of the config file requires the path module, which will help resolve all path-related functions.

    b) Next, you will notice the entry field. This field simply tells webpack where to find the starting/main script. webpack will use this file as a starting point, and then recursively walk through each import dependency to link all files relating to the entry file.

    c) The next field is output, and this tells webpack where to save the bundled file. In our example, we are saving bundled files to the dist folder under the name bundle.js (remember we referenced bundle.js in our HTML file).

    d) Finally, in the module field, we specify that we want to transpile each script using Babel, and we also exclude transpiling node_modules. With this webpack configuration file, you're ready to transpile and bundle your source code.

  8. In your package.json file, you'll add a script command that will call webpack, as shown in the following code block:
    {
      ...
      "scripts": {
        "build": "webpack --mode=production",
        "test": "echo \"Error: no test specified\" && exit 1"
      },
     
      ...
    }
  9. In your terminal, run the following command:
    $ npm run build

    This command will call the build script in your package.json file, and this, in turn, will ask webpack to bundle your code referencing the config file you created earlier.

    Following successful compilation, you should have the following output in your terminal:

Figure 1.2 – webpack bundling successful output

Figure 1.2 – webpack bundling successful output

Upon successful completion of the preceding steps, you can navigate to the dist folder where you will find an extra file – bundle.js. This file has already been referenced by the index.html file, and as such will be executed anytime we load the index.html file in the browser.

To test this, open the index.html file in your default browser. This can be done by navigating to the directory and double-clicking on the index.html file.

Once you have the index.html file opened in your browser, you should open the developer console, where you can find your code output, as in the following screenshot:

Figure 1.3 – index.js output in the browser console

Figure 1.3 – index.js output in the browser console

This shows that you have successfully transpiled and bundled your modern JS code into a format that can be executed in any browser, be it old or new.

Further reading

Bundling files can quickly become difficult and confusing, especially as the project gets bigger. If you require further understanding of how to bundle files, you can reference the following resources:

* Getting Started (https://webpack.js.org/guides/getting-started/) with webpack

* Usage Guide (https://babeljs.io/docs/en/usage) for Babel

* How to enable ES6 (and beyond) syntax with Node and Express (https://www.freecodecamp.org/news/how-to-enable-es6-and-beyond-syntax-with-node-and-express-68d3e11fe1ab/)

In the next section, you'll learn how to set up testing and perform unit testing in your JavaScript application.