Book Image

Node Cookbook - Third Edition

By : David Mark Clements, Mathias Buus Madsen, Peter Elger, Matteo Collina
Book Image

Node Cookbook - Third Edition

By: David Mark Clements, Mathias Buus Madsen, Peter Elger, Matteo Collina

Overview of this book

Today's web demands efficient real-time applications and scalability. Asynchronous event-driven programming is ideal for this, and this is where Node.js comes in. Server-side JavaScript has been here since the 90s, but Node got it right. With Node for tooling and server-side logic, and a browser-based client-side UI, everything is JavaScript. This leads to rapid, fluid development cycles. The full-stack, single language experience means less context-switching between languages for developers, architects and whole teams. This book shows you how to build fast, efficient, and scalable client-server solutions using the latest versions of Node. The book begins with debugging tips and tricks of the trade, and how to write your own modules. Then you'll learn the fundamentals of streams in Node.js, discover I/O control, and how to implement the different web protocols. You'll find recipes for integrating databases such as MongoDB, MySQL/MariaDB, Postgres, Redis, and LevelDB. We also cover the options for building web application with Express, Hapi and Koa. You will then learn about security essentials in Node.js and advanced optimization tools and techniques. By the end of the book you will have acquired the level of expertise to build production-ready and scalable Node.js systems. The techniques and skills you will learn in this book are based on the best practices developed by nearForm, one of the leaders in Node implementations, who supported the work of the authors on this book.
Table of Contents (12 chapters)

Processing Big Data

Let's dive right into it by looking at a classic Node problem: counting all Node modules available on npm. The npm registry exposes an HTTP endpoint where we can get the entire contents of the npm registry content as JSON.

Using the command line tool, curl, which is included (or at least installable) on most operating systems, we can try it out.

$ curl https://skimdb.npmjs.com/registry/_changes?include_docs=true

This will print a new line delimited JSON stream of all modules.

The JSON stream returned by the registry contains a JSON object for each module stored on npm followed by a new line character.

A simple Node program that counts all modules could look like this:

var request = require('request') 
var npmDb = 'https://skimdb.npmjs.com'
var registryUrl = `${npmDb}/registry/_changes?include_docs=true`
request(registryUrl, function (err...