Planning is essential in order to achieve the best results possible. High performance is built from the ground up and starts with how you plan and develop. It obviously depends on physical resources, as you can't perform well when you don't have sufficient memory to accomplish your task, but it also depends greatly on how you plan and develop an application. Mastering tools will give much better performance chances than just using them.
Setting the bar high from the beginning of development will force the planning to be more prudent. Some bad planning of the database layer can really downgrade performance. Also, cautious planning will cause developers to think more about use cases and program more consciously.
High performance is when you have to think about a new set of resources (processor, memory, storage) because all that you have is exhausted, not just because one resource is. A high-performance application shouldn't need a second server when a little processor is used and the disk is full. In such a case, you just need bigger disks.
Applications can't be designed as monolithic these days. An increasing user base enforces a distributed architecture, or at least one that can distribute load by having multiple instances. This is very important to accommodate in the beginning of the planning, as it will be harder to change an application that is already in production.
Most common applications will start performing worse over time, not because of deficit of processing power but because of increasing data size on databases and disks. You'll notice that the importance of memory increases and fallback disks become critical to avoiding downtime. It's very important that an application be able to scale horizontally, whether to shard data across servers or across regions.
A distributed architecture also increases performance. Geographically distributed servers can be more closed to clients and give a perception of performance. Also, databases distributed by more servers will handle more traffic as a whole and allow DevOps to accomplish zero downtime goals. This is also very useful for maintenance, as nodes can be brought down for support without affecting the application.
To know whether an application performs well or not under specific environments, we have to test it. This kind of test is called a benchmark. Benchmarking is important to do and it's specific to every application. Even for the same language and platform, different applications might perform differently, either because of the way in which some parts of an application were structured or the way in which a database was designed.
Analyzing the performance will indicate bottleneck of your application, or if you may, the parts of the application that perform not good as others. These are the parts that need to be improved. Constantly trying to improve the worst performing parts will elevate the application's overall performance.
There are plenty of tools out there, some more specific or focused on JavaScript applications, such as benchmarkjs (http://benchmarkjs.com/) and ben (https://github.com/substack/node-ben), and others more generic, such as ab (http://httpd.apache.org/docs/2.2/programs/ab.html) and httpload (https://github.com/perusio/httpload). There are several types of benchmark tests depending on the goal, they are as follows:
Load testing is the simplest form of benchmarking. It is done to find out how the application performs under a specific load. You can test and find out how many connections an application accepts per second, or how many traffic bytes an application can handle. An application load can be checked by looking at the external performance, such as traffic, and also internal performance, such as the processor used or the memory consumed.
Soak testing is used to see how an application performs during a more extended period of time. It is done when an application tends to degrade over time and analysis is needed to see how it reacts. This type of test is important in order to detect memory leaks, as some applications can perform well in some basic tests, but over time, the memory leaks and their performance can degrade.
Spike testing is used when a load is increased very fast to see how the application reacts and performs. This test is very useful and important in applications that can have spike usages, and operators need to know how the application will react. Twitter is a good example of an application environment that can be affected by usage spikes (in world events such as sports or religious dates), and need to know how the infrastructure will handle them.
All of these tests can become harder as your application grows. Since your user base gets bigger, your application scales and you lose the ability to be able to load test with the resources you have. It's good to be prepared for this moment, especially to be prepared to monitor performance and keep track of soaks and spikes as your application users start to be the ones responsible for continuously test load.
Because of this continuous demand of performant applications, composition becomes very important. Composition is a practice where you split the application into several smaller and simpler parts, making them easier to understand, develop, and maintain. It also makes them easier to test and improve.
Avoid creating big, monolithic code bases. They don't work well when you need to make a change, and they also don't work well if you need to test and analyze any part of the code to improve it and make it perform better.
The Node.js platform helps you—and in some ways, forces you to—compose your code. Node.js Package Manager (NPM) is a great module publishing service. You can download other people's modules and publish your own as well. There are tens of thousands of modules published, which means that you don't have to reinvent the wheel in most cases. This is good since you can avoid wasting time on creating a module and use a module that is already in production and used by many people, which normally means that bugs will be tracked faster and improvements will be delivered even faster.
The Node.js platform allows developers to easily separate code. You don't have to do this, as the platform doesn't force you to, but you should try and follow some good practices, such as the ones described in the following sections.
Don't rewrite code unless you need to. Take your time to try some available modules, and choose the one that is right for you. This reduces the probability of writing faulty code and helps published modules that have a bigger user base. Bugs will be spotted earlier, and more people in different environments will test fixes. Moreover, you will be using a more resilient module.
One important and neglected task after starting to use some modules is to track changes and, whenever possible, keep using recent stable versions. If a dependency module has not been updated for a year, you can spot a problem later, but you will have a hard time figuring out what changed between two versions that are a year apart. Node.js modules tend to be improved over time and API changes are not rare. Always upgrade with caution and don't forget to test.
Again, you should always split your code into smaller parts. Node.js helps you do this in a very easy way. You should not have files bigger than 5 kB. If you have, you better think about splitting it. Also, as a good rule, each user-defined object should have its own separate file. Name your files accordingly:
// MyObject.js
module.exports = MyObject;
function MyObject() {
// …
}
MyObject.prototype.myMethod = function () { … };Another good rule to check whether you have a file bigger than it should be; that is, it should be easy to read and understand in less than 5 minutes by someone new to the application. If not, it means that it's too complex and it will be harder to track and fix bugs later on.
The platform is designed to be asynchronous, so you shouldn't go against it. Sometimes, it can be really hard to make some recursive tasks or even simply cycle through a list of tasks that have to run serially. You should avoid creating a module to handle asynchronous tasks, as there are some used and tested by hundreds of thousands of people out there. For instance, async is a simple and very practical way of helping the developer perform better, and the learning curve is very smooth:
async.each(users, function (user, next) {
// do something on each user object
return next();
}, function (err) {
// done!
});This module has a lot of methods similar to the ones you find in the array object, such as map, reduce, filter, and each, but for iterating asynchronously. This is extremely useful when your application gets more complex and some user actions require some serialized tasks. Error handling is also done correctly and the execution stop is done as expected. The module helps run serial or parallel tasks.
Also, serial tasks that would usually enforce a developer to nest calls and enter the callback hell can simply be avoided. This is especially useful when, for example, you need to perform a transaction on a database with several queries involved.
Another common mistake when writing asynchronous code is throwing errors. Callbacks are called outside the scope where they are defined, and so you cannot just put the callback inside a try/catch block. Therefore, avoid doing this unless it's a very critical error that should make your application stop and quit. In Node.js, throwing an exception without catching it will trigger an uncaughtException event.
The platform has a rule that is consensual for most developers—the so-called error-first callback style. This rule is of extreme importance, since it allows an easier reuse of your code. Even if you have a function where there's no chance of throwing an error, or when you just don't want it to throw and use some kind of error handling inside the function, your callback should always reserve the first argument for an error event if it's always null. This will allow your function to be used with an async module. Also, other developers will be counting on this style when debugging, so always reverse the first argument as an error object.
Plus, you should always reserve the last argument of the function as the callback. Never define arguments after your callback:
function mySuperFunction(arg1, ..., argN, next) {
// do some voodoo
return next(null, my_result); // 1st argument reserved for error
}Library functions are another type of module you should use. They help in handling repetitive tasks, and every developer has to perform such tasks. Some of these repetitive tasks can be done with no effort, just by using a library function from lodash or underscore. They are an important part of your code and have good optimizations that you don't even have to think about. Many cycling tasks, such as finding an object in an array based on an object key, or mapping an array of objects to an array of keys of every object, are one-liners in these libraries. Read the documentation first to avoid using the library and not fully using its potential.
Although these kinds of modules can be useful, they can also downgrade performance if they are not chosen well. Some modules are designed to help developers in some tasks, but do not target performance—just convenience. In other words, these modules can help you develop faster, but you shouldn't forget the complexity of each function. Otherwise, you will be calling the same function several times because you forget about its complexity, instead of calling it once and saving the results.
Functions are very important in this platform. This is no surprise since the language is functional and has first-class functions. There are some rules you should follow when writing functions that will make your life easier when debugging or optimizing it later. They also avoid some errors as they try to enforce some common structure. Once again, you can enforce these rules using, for example, JSCS (http://jscs.info/):
Always name your functions, especially when they're closures used as callbacks. This allows you to identify them in stack traces when your code breaks. Also, they allow a new developer to rapidly know what the function is supposed to do. Still, avoid long names:
socket.on("data", function onSocketData(data) { // … });Don't nest your conditions, and return as early as possible. If you have a condition that must return something in a function and if you return, you don't have to use the
elsestatement. You also avoid a new indent level, reducing your code and simplifying its revision. If you don't do this, you will end up in a condition hell, with several levels if you have two or more conditions to satisfy:// do this if (someCondition) { return false; } return someThing; // instead of this: if (someCondition) { return false; } else { return someThing; }Create small and simple functions. Don't span your functions for more lines than your screen can handle. Even if your task cannot be reused, split the function into smaller ones. It is even better to put it into a new module and publish it. In this way, you can reuse them at the frontend if you need them. This can also allow the engine to optimize some smaller functions when it is unable to optimize the previous big function. Again, this is important if you don't want a developer to be reading your application code for a week or two before being able to touch anything.
Testing your modules is a hard job and is usually neglected, but it's very important to make tests for your modules. The first ones are the hard ones. Look for a test tool that you like, such as vows, chai, or mocha. If you don't know how to start, read a module's documentation, or another module's test code. But don't give up on testing.
Note
If you need help, read the test tools' websites mentioned earlier, as they usually help you get started. Alternatively, you can take a look at Igor's post (https://semaphoreci.com/community/tutorials/getting-started-with-node-js-and-mocha)at semaphore.
After you start adding one or two tests, more will follow. One big advantage of testing your module from the beginning is that when you spot a bug, you can make a test case for it, to be able to reproduce it and avoid it in the future.
Code coverage is not crucial but can help you see how your tests cover your module code base, and if you're just testing a small part. There are some coverage modules, such as istanbul or jscoverage; choose the one that works best for you. Code coverage is done together with testing, so if you don't test it, you won't be able to see the coverage.
As you might want to improve the performance of an application, every dependency module should be looked at for improvements. This can be done only if you test them. Dependency version management is of great importance, and it can be hard to keep track of new versions and changes, but they might give you some good news. Sometimes, modules are refactored and performance is boosted. A good example of this is database access modules.