Book Image

Node Cookbook - Third Edition

By : David Mark Clements, Mathias Buus Madsen, Peter Elger, Matteo Collina
Book Image

Node Cookbook - Third Edition

By: David Mark Clements, Mathias Buus Madsen, Peter Elger, Matteo Collina

Overview of this book

Today's web demands efficient real-time applications and scalability. Asynchronous event-driven programming is ideal for this, and this is where Node.js comes in. Server-side JavaScript has been here since the 90s, but Node got it right. With Node for tooling and server-side logic, and a browser-based client-side UI, everything is JavaScript. This leads to rapid, fluid development cycles. The full-stack, single language experience means less context-switching between languages for developers, architects and whole teams. This book shows you how to build fast, efficient, and scalable client-server solutions using the latest versions of Node. The book begins with debugging tips and tricks of the trade, and how to write your own modules. Then you'll learn the fundamentals of streams in Node.js, discover I/O control, and how to implement the different web protocols. You'll find recipes for integrating databases such as MongoDB, MySQL/MariaDB, Postgres, Redis, and LevelDB. We also cover the options for building web application with Express, Hapi and Koa. You will then learn about security essentials in Node.js and advanced optimization tools and techniques. By the end of the book you will have acquired the level of expertise to build production-ready and scalable Node.js systems. The techniques and skills you will learn in this book are based on the best practices developed by nearForm, one of the leaders in Node implementations, who supported the work of the authors on this book.
Table of Contents (12 chapters)

Debugging Node with Chrome Devtools

Node 6.3.0 onwards provides us with the --inspect flag, which we can use to debug the Node runtime with Google Chrome's Devtools.

Debugging legacy Node
This recipe can be followed with older versions of Node prior to Node 6.3.0 – it just requires a little more set up. To follow this recipe with a legacy version of Node, jump to Using node-inspector with older Node versions in the There's more... section of this recipe first.

In this recipe, we're going to diagnose and solve a problem in a simple Express application.

Getting ready

We're going to debug a small web server, so let's create that real quick.

On the command line, we execute the following commands:

$ mkdir app
$ cd app
$ npm init -y
$ npm install --save express
$ touch index.js future.js past.js
  

Our index.js file should contain the following:

const express = require('express') 
const app = express() 
const past = require('./past') 
const future = require('./future') 
 
app.get('/:age', (req, res) => { 
  res.send(past(req.params.age, 10) + future(req.params.future, 10)) 
}) 
 
app.listen(3000) 

Our past.js file should look like this:

module.exports = (age, gap) => { 
  return `${gap} years ago you were ${Number(age) - gap}<br>` 
} 

And our future.js file should be as follows:

module.exports = (age, gap) => {
return `In ${gap} years you will be ${Number(age) + gap}<br>`
}
Web frameworks
We're only using Express here as an example. To learn more about Express and other frameworks, see Chapter 7, Working With Web Frameworks.

How to do it...

When we run our server (which we created in the Getting ready section), and navigate our browser to http://localhost:3000/31 , the output is as follows:

10 years ago you were 21 
In 10 years you will be NaN 

It looks like we have a Not a Number problem.

Let's start our server in inspection mode:

$ node --inspect index.js

This will output a message that the debugger is listening.

We can connect to the debugger using the Chrome browser. Let's open Chrome and navigate to chrome://inspect.

Ensuring that we're in the Devices tab, we should be able to see our Node process underneath the Remote Target section, as in the following screenshot:

We should then see something like the following:

The module wrapper
Notice that the Devtools Code section shows an additional outer function wrapping the code we placed into index.js. This outer function is added at runtime to each code file loaded into the process (either by directly starting the file with node or by using require to load it). This outer function is the module wrapper, it's the mechanism Node uses to supply local references like module and __filename that are unique to our module without polluting global scope.

Now let's set a breakpoint inside our route handler, on line 7.

If we click the number 7 in the LOC column to the left of the code, an arrow shape will appear over and around the number (which will turn white). Over in the right-hand column, in the Breakpoints pane we should also see a checkbox with index.js:7 next to it, while beneath that is the code from the line we've set a breakpoint on.

In short, the Devtools GUI should now look something like the following:

Now let's open a new tab and navigate to http://localhost:3000/31:

This will cause the breakpoint to trigger, and Devtools will immediately grab focus.

The next thing we see should look like the following:

We can see line 7 is now highlighted, and there's a sort of tooltip showing us the values of the req and res objects on the line above.

Over in the right column, the Call Stack panel is full of call frames (the functions in the stack), and there's now a blue play button in the control icons at the top of the right column. If we were to scroll the right column, we'd also see the Scope pane is populated with references.

The debugger is waiting for us to allow execution to proceed, and we can chose whether to step over, in, or out of the next instruction.

Let's try stepping in. This is the down arrow pointing to a dot, the third icon from the left in the controls section:

When we press this, we step into the past function, which is in the past.js file, so Devtools will open a new tab in the center code panel and highlight the line that is about to execute (in our case, line 2):

So let's step out of the past function by pressing the arrow pointing up and away from a dot, next to the step-in icon:

The second line of the output seems to have the issue, which is our future function.

Now that we've stepped out, we can see that the call to future is highlighted in a darker shade of blue:

Now let's press the step-in icon again, which will take us into the future function in the future.js file:

Okay, this is the function that generates that particular sentence with the NaN in it. A NaN can be generated for all sort of reasons, such as dividing zero by itself, subtracting Infinity from Infinity, trying to coerce a string to a number when the string does not hold a valid number, to name a few. At any rate, it's probably something to do with the values in our future function.

Let's hover over the gap variable. We should see the following:

Seems fine. Now let's hover over the age variable:

Wait... why does that say undefined ? We vicariously passed 31 by navigating to http://localhost:3000/31.

To be sure our eyes aren't deceiving us, we can double-check by collapsing the Call Stack section (by clicking the small downwards arrow next to the C of Call Stack). This will make room for us to easily see the Scope section, which reports that the age variable is indeed undefined, as in the following screenshot:

Well, Number(undefined) is NaN, and NaN + 10 is also NaN.

Why is age set to undefined?

Let's open up the Call Stack bar again and click the second row from the top (which says app.get).

We should be back in the index.js file again (but still frozen on line 2 of future.js), like so:

Now let's hover over the value we're passing in to future:

That's undefined too. Why is it undefined?

Oh. That should be req.params.age, not req.params.future. Oops.

To be absolutely sure, let's fix it while the server is running. If we hit the blue play button once we should see something like this:

Now let's click line 7 again to remove the breakpoint. We should be seeing:

Now if we click immediately after the e in req.params.future we should get a blink cursor. We backspace out the word future and type the word age, making our code look like this:

Finally, we can live save those changes in our running server by pressing Cmd + S on macOS, or Ctrl + S on Windows and Linux.

Finally, let's check our route again:

OK, we've definitely found the problem, and verified a solution.

How it works...

We don't really need to know how debugging Node with devtools is made possible in order to avail ourselves of the tool, however, for the curious here's a high-level overview.

Debugging ability is ultimately provided by V8, the JavaScript engine used by Node. When we run node with the --inspect flag, the V8 inspector opens a port that accepts WebSocket connections. Once a connection is established, commands in the form of JSON packets are sent back and forth between the inspector and a client.

The chrome-devtools:// URI is a special protocol recognized by the Chrome browser that loads the Devtools UI (which is written in HTML, CSS, and JavaScript, so can be loaded directly into a normal browser tab). The Devtools UI is loaded in a special mode (remote mode), where a WebSocket endpoint is supplied via the URL.

The WebSocket connection allows for bi-directional communication between the inspector and the client. The tiny Inspector WebSocket server is written entirely in C and runs on a separate thread so that when the process is paused, the inspector can continue to receive and send commands.

In order to achieve the level of control we're afforded in debug mode (ability to pause, step, inspect state, view callstack, live edit) V8 operations are instrumented throughout with Inspector C++ functions that can control the flow, and change state in place.

For instance, if we've set a breakpoint, once that line is encountered, a condition will match in the C++ level that triggers a function that pauses the event loop (the JavaScript thread). The Inspector then sends a message to the client over the WebSocket connection telling it that the process is paused on a particular line and the client updates its state. Likewise, if the user chooses to step into a function, this command is sent to the Inspector, which can briefly unpause and repause execution in the appropriate place, then sends a message back with the new position and state.

There's more...

Let's find out how to debug older versions of Node, make a process start with a paused runtime and learn to use the built-in command-line debugging interface.

Using node-inspector with older Node versions

The --inspect flag and protocol were introduced in Node 6.3.0, primarily because the V8 engine had changed the debugging protocol. In Node 6.2.0 and down, there's a legacy debugging protocol enabled with the --debug flag, but this isn't compatible with the native Chrome Devtools UI.

Instead, we can use the node-inspector tool, as a client for the legacy protocol.

The node-inspector tool essentially wraps an older version of Devtools that interfaces with the legacy debug API, and then hosts it locally.

Let's install node-inspector:

$ npm i -g node-inspector

This will add a global executable called node-debug, which we can use as shorthand to start our process in debug mode.

If we could run our process like so:

$ node-debug index.js

We should see output that's something like the following:

Node Inspector v0.12.10
Visit http://127.0.0.1:8080/?port=5858 to start debugging.
Debugging `index.js`
    
Debugger listening on [::]:5858

When we load the URL http://127.0.0.1:8080/?port=5858 in our browser we'll again see the familiar Devtools interface.

By default, the node-debug command start our process in a paused state. After pressing run (the blue play button), we should now be able to follow the main recipe in it's entirely using a legacy version of Node.

Immediately pausing a process on start

In many cases, we want to debug a process from initialization, or we may want to set up breakpoints before anything can happen.

From Node 8 onwards, we use the following to start Node in an immediately paused state:

$ node --inspect-brk index.js

In Node 6 (at time of writing, 6.10.0), --inspect is supported but --inspect-brk isn't. Instead, we can use the legacy --debug-brk flag in conjunction with --inspect like so:

$ node --debug-brk --inspect index.js

In Node v4 and lower, we'd simply use --debug-brk instead of --debug (in conjunction with another client, see Using Node Inspector with older Node versions)

node debug

There may be rare occasions when we don't have easy access to a GUI. In these scenarios, command-line abilities become paramount.

Let's take a look at Nodes, built in command-line debugging interface.

Let's run our app from the main recipe like so:

$ node debug index.js

When we enter debug mode, we see the first three lines of our entry point (index.js).

Upon entering debug mode, the process is paused on the first line of the entry point. By default, when a breakpoint occurs the debugger shows two lines before and after the current line of code, since this is the first line we only see two lines after.

The debug mode provides several commands in the form of functions, or sometimes as magic getter/setters (we can view these commands by typing help and hitting Enter).

Let's get a little context using the list function:

debug> list(10) 

This provides 10 lines after our current line (again it would also include 10 lines before, but we're on the first line so there's no prior lines to show).

We're interested in the seventh line, because this is the code that's executed when the server receives a request.

We can use the sb function (which stands for Set Breakpoint) to set a break point on line 7, like so:

debug> sb(7)  

Now if we use list(10) again, we should see an asterisk (*) adjacent to line 7:

debug> list(10)  

Since our app began in paused mode, we need to tell the process to begin running as normal so we can send a request to it.

We use the c command to tell the process to continue, like so:

debug> c 

Now let's make a request to our server, we could use a browser to do this, or if we have curl on our system, in another terminal we could run the following:

$ curl http://localhost:3000/31 

This will cause the process to hit our breakpoint and the debugger console should print out break in index.js:7 along with the line our code is currently paused on, with two lines of context before and after. We can see a right caret (>) indicating the current line:

Now let's step in to the first function. To step in, we use the step command:

debug> step

This enters our past.js file, with the current break on line 2.

We can print out references in the current debug scope using the exec command, let's print out the values of the gap and age arguments:

debug> exec gap
debug> exec age

Everything seems to be in order here.

Now let's step back out of the past function. We use the out command to do this, like so:

debug> out 

We should now see that the future function is a different color, indicating that this is the next function to be called. Let's step into the future function:

debug> step

Now we're in our future.js file, again we can print out the gap and age arguments using exec:

debug> exec gap
debug> exec age

Aha, we can see that age is undefined. Let's step back up into the router function using the out command:

debug> out 

Let's inspect req.params.future and req.params:

debug> req.params.future
debug> req.params  

It's now (again) obvious where the mistake lies. There is no req.params.future; that input should be req.params.age.

See also

  • Creating an Express web app, in Chapter 7, Working with Web Frameworks
  • Writing module code, in Chapter 2, Writing Modules
  • Profiling memory, in Chapter 9, Optimizing Performance
  • CPU profiling with Chrome Devtools, in the There's more... section of Finding Bottlenecks with Flamegraphs, in Chapter 9, Optimizing Performance