MongoDB, Express, Angular, and Node.js Fundamentals

The MEAN architecture is comprised of four different JavaScript technologies when they are stacked successively. The architecture is designed so that each component performs specific tasks individually as well as in sync with other layers in the stack to achieve an end goal.

In the MEAN architecture that's shown in the following diagram, the client initiates the requests, which flow right from the first layer down to the last. Appropriate responses to those requests then flow right through all the layers back up to the client.

The client first interacts with the user interface, which is built using the Angular frontend framework. Requests made on the interface are passed to Node.js, the server-side engine. Then, the middleware framework known as Express.js makes a request to MongoDB, which is the database. Express.js retrieves the response in the form of data from the database. Finally, this response is then returned by Node.js to the client via the user display:

Figure 1.2: Workings of the MEAN architecture

There are several advantages to using the various MEAN components, which has made it a technology stack to reckon with for start-ups and big companies alike. There are several reasons why the MEAN stack is preferred in web application development. A few of these reasons are listed here:

Owing to its performance benefits, the MEAN stack has been adopted by many contemporary commercial organizations, such as Uber, PayPal, Netflix, and so on to build their websites and web applications. Ulbora and Trakit.me are a couple of other web applications that are built using the MEAN stack.

Before going into MEAN application development, we will briefly discuss the individual technologies that make up the MEAN stack. We will start with MongoDB.

MongoDB

MongoDB is a document-oriented database program. The database, which is classified as NoSQL, is developed by MongoDB Inc. It's a free and open source database that uses JSON-like documents with schemas. Described here are some of the major features that make MongoDB stand out in application development:

Strong consistency: Strong consistency is a default feature in MongoDB, and thus it takes no additional upgrades to create consistency. It's also highly guaranteed: as long as a write is successful, you will be able to read it. Automatic failover of the replica set makes it highly available. Also, flexibility and scalability are not readily available in traditional databases that only rely on a defined schema. In comparison, MongoDB, which is NoSQL, relies on the key-to-value pair approach. This enables flexibility and scalability in MongoDB.

Real-time aggregation: This is a built-in MongoDB feature that allows real-time statistical analysis and the generation of pre-aggregated reports on request.

Ad hoc queries: These are impromptu queries that are written to extract information from the database, but only when necessary.

Indexing: Indexing in MongoDB makes it easy to search documents without performing a collection-wide scan by matching specific queries with documents.

Database-as-a-service: Database-as-a-service offerings by MongoDB enable users to access databases without having to install software or set up hardware. MongoDB Atlas, which is a cloud-hosted service offering by MongoDB Inc., will be used in this book. As we progress in the next chapter, we will go through all the processes involved in its setup.

Despite having several advantages, there are a few limitations of MongoDB that we need to be aware of:

Limited data size: Documents have a maximum data size of 16 MB.

Limited nesting: The maximum number of documents that can be nested is 100.

Joins not supported: This is one of the main differences between RDBMS and NoSQL. MongoDB is not designed to have related columns. Rather, it stores de-normalized data and thus joins are not supported.

However, these limitations cannot overshadow the fact that MongoDB provides high performance, high availability, and automatic scaling, thus mitigating—if not completely eliminating—the preceding limitations.

So far, we have introduced the MEAN stack and its database component, MongoDB. We will now continue with the introduction of the second component, Express.js—a middleware application framework for developing web applications.

Express.js or simply Express is a Node.js web application framework that provides a strong set of features for web and mobile applications. It's free, open source, fast, un-opinionated (that is, there is no "right way" to complete a certain task in Express), and is comprised of a simple web framework that's designed for building web applications and APIs. The framework has achieved a dominant position in the industry as the server framework for Node.js. Express possesses rich features and advantages. A few of these are listed as follows:

The preceding list includes a few of the most dominant features and advantages of Express, and each of them contribute to the performance benefits afforded by this framework. In Express, third-party plugins (middleware) can be set up to respond to HTTP/RESTful requests, and a routing table can also be defined to perform different HTTP operations. HTML pages are dynamically rendered by Express based on passing arguments to templates. Express supports a wide range of templating languages, such as Pug, React, hbs, and h4e. Furthermore, it has detailed documentation to help developers maximize its uses. The community support is huge: as of now, Express has gathered over 30,000 GitHub stars, which means that it is well accepted and supported in the open source community. Also, Express allows for the addition of custom code. In terms of performance, Express leads other frameworks because of its inherent properties that make the routing process efficient and faster.

Express presents a thin layer that enables the addition of robust features and functionalities to your web applications. It organizes web applications into a model view controller (MVC) architecture and manages everything from routes, to rendering views, and pre-forming HTTP requests.

However, we do need to be aware of certain limitations while developing apps in Express. In a real-life situation, when projects become more complex and code base size increases, refactoring becomes very complex and a lot of manual labor is required. This is because the absence of organization makes the handling of complex applications difficult as Express is not designed to manage complexity by itself. Furthermore, the cost of scaling increases with application complexity.

In this section, we introduced another MEAN stack component – Express.js. We described its features and viewed popular Node.js frameworks that have been built using Express.js. In the next section, we will continue with the introduction of the third component, Angular.

The next component after Express in the MEAN stack is a JavaScript frontend framework known as Angular. We will be introducing Angular features across all of its existing versions, and then we will talk briefly about its advantages and limitations.

Angular is an open source TypeScript and JavaScript-based frontend web application structural framework and is managed and sustained by Google. Being open source, a community of software developers and corporations provide maintenance and address issues that are encountered when developing single-page applications. Listed here are some of the advantages that make Angular stand out:

However, note that no framework is perfect and there have been several Angular versions. The following list shows how Angular has evolved over the years, along with its different versions:

However, you should be aware that no web framework is perfect, and in the case of Angular, its verbosity and complexity renders it a difficult framework to master. Furthermore, Angular has a steep learning curve.

In this section, we introduced the third component of the MEAN stack – Angular. We discussed its features, as well as its limitations. In the next section, we will introduce our final MEAN component, a server-side technology known as Node.js.

Node is an open source program that is built on a JavaScript runtime engine (Chrome's V8) and consists of JavaScript libraries that are implemented in the form of modules. The cross-platform server environment is lightweight and efficient because it uses an event-driven, non-blocking I/O model. For example, PayPal employs Node to solve the problem that arises owing to the boundary between the browser (client side) and server (server side) by enabling both to be written in JavaScript. This, in turn, helps PayPal unify their engineering specialties into one team, thus enabling them to focus on user needs and create solutions to meet these on the technology stack level. Some other websites and applications built on Node are eBay, GoDaddy, Groupon, Uber, Netflix, Mozilla, Yahoo, LinkedIn, and so on. The outstanding performance offered by Node makes it the heart of the MEAN stack. According to a number of benchmarks, JavaScript technology for the backend outperforms several other backend programming languages, such as PHP, Python, and Ruby. Listed here are some of the major advantages of Node:

However, you should be aware of a few points before beginning application development in Node. Using relational databases in conjunction with Node might be a daunting task because Node does not provide support for them. Also, using Node is unsuitable for heavy-computation applications. Having given an overview of Node, we will now progress and build our first Node application in the next section.

Before beginning this exercise, make sure you have Node installed. Our aim is to create a Node program that will print Hello World on the browser window once the appropriate command is passed on to the server. To do so, the following steps have to be performed:

One of the learning objectives of this book is to create the components of a blogging application with basic features. We will commence its development in this chapter. Most of this will be done in the activity section of each topic.

You have been tasked with developing the HTTP server component for a blogging application. Your aim is to call the server and listen to the localhost on a specified port. Before you begin this activity, ensure that you have completed the previous exercise, in addition to creating a project directory named Blogging Application and a subfolder named Server. You can use an IDE of your choice; however, in this book, we are using Visual Studio.

To complete this activity, the following steps have to be completed:

Node is built to be asynchronous in everything that it does. In this view, a callback is an asynchronous equivalent of a function that is called after a given task is completed. Alternatively, it can be defined as a function that is passed into another function so that the latter can call it on the completion of a given task. It allows other programs to keep running, thereby preventing blocking.

Let's consider a JavaScript global function, setTimeout(), as implemented in the following snippet:

setTimeout(function () {
console.log("1…2…3…4…5 secs later.");
}, 5000);
setTimeout() accepts a callback function and a delay in milliseconds as first and second arguments, respectively. The callback function is fired after 5,000 milliseconds has elapsed, thereby printing "1…2…3…4…5 secs later." to the console.
An interesting thing is that the preceding code can be rewritten and simplified, as shown here:
var callback = function () {
console.log("1…2…3…4…5 secs later.");
};
setTimeout(callback, 5000)
Another example of callbacks can be seen in filesystem operations. readFile (asynchronous) and readFileSync (synchronous) are two unique API functions in the Node library that can be used to read a file.
An example of synchronous reading is as follows:
var Filedata = fs.readFileSync('fileText.txt');
console.log(FileData);

The readFileSync() method reads a file synchronously (all of the content is read at the same time). It takes in the file path (this could be in the form of a string, URL, buffer, or integer) and an optional parameter (either an encoder, which could be a string, null, or a flag in the form of a string) as an argument. In the case of the preceding snippet, the synchronous filesystem function takes in a file (fileText.txt) from the directory and the next line prints the contents of the file as a buffer. Note that if the encoding option is specified, then this function returns a string. Otherwise, it returns a buffer, just as we've seen here.

An example of asynchronous reading is as follows:

var callback = function (err, FileData) {
if (err) return console.error(err);
console.log(FileData);
};
fs.readFile('fileText.txt', callback);

In the preceding snippet, the readFile() method asynchronously reads the entire contents of a file (read serially until all its content is entirely read). It takes in the file path (this could be in the form of a string, URL, buffer, or integer), an optional parameter (either an encoder, which could be a string or null, or a flag in the form of a string), and a callback as arguments. It can be seen from the first line that the callback function accepts two arguments: err and data (FileData). The err argument is passed first because, as API calls are made, it becomes difficult to track an error, thus, it's best to check whether err has a value before you do anything else. If so, stop the execution of the callback and log the error. This is known as error-first callback.

In addition, if callbacks are not used (as seen in the previous example on synchronous reading) when dealing with a large file, you will be using massive amounts of memory, and this leads to a delay before the transfer of data begins (network latency). To summarize, from the use of the two filesystem functions, that is, the readFileSync(), which works synchronously, and the readFile(), which works asynchronously, we can deduce that the latter is safer than the former.

Also, in a situation where a callback is heavily nested (multiple asynchronous operations are serially executed), callback hell (also known as the pyramid of doom) may occur, when the code becomes unreadable and maintenance becomes difficult. Callback hell can be avoided by breaking callbacks into modular units (modularization) by using a generator with promises, implementing async/await, and by employing a control flow library.

An event loop is an efficient mechanism in Node that enables single-threaded execution. Everything that happens in Node is a reaction to an event. So, we can say that Node is an event-based platform, and that the event loop ensures that Node keeps running. Node uses event loops similar to a FIFO (first-in first-out) queue to arrange the tasks it has to do in its free time.

Let's assume that we have three programs running simultaneously. Given that each program is independent of another at any given time, they won't have to wait for the other before the output of each program is printed on the console. The following diagram shows the event loop cycle:

Figure 1.5: Event loop cycle

The following diagram shows the event queue and the call stack operations:

Figure 1.6: Event queue and call stack

To understand the concept of an event loop, we will consider the following implementation of an event loop in a program to calculate a perimeter:

const perimeter = function(a,b){
return 2*(a+b);
}
constshowPerimeter = function(){
console.log(perimeter(2,2));
}
const start = function(){
console.log('Initailizing.....');
console.log('Getting Started.....');
setTimeout(showPerimeter,10000);
console.log('Ending program.....');
console.log('Exiting.....');
}
start();
The operation sequence of the preceding snippet is listed here:
start() is pushed to memory, which is known as a call stack. 
console.log ('Initializing …..') is pushed and popped on execution. 
console.log ('Getting Started …..') is pushed and popped on execution.
setTimeout(showPerimiter,10000) is pushed into the stack, which is where a timer is created by the API, and the program does not wait for the callback. 
console.log ('Ending program…') is pushed and popped on execution.
If 10 seconds elapse, the showPerimiter(2,2) callback function will be sent to the event queue to wait until the stack is empty. 
console.log ('Exiting program…') is pushed and popped on execution.
Executed.start() is also taken out of the stack. 
The showPerimiter(2,2) function is moved to the stack and is executed. 
Finally, 8 is printed to the command line, which is the perimeter.
Now, we are able to understand the concept of the event loop in Node. Next, we will explore the EventEmitter, which is one of the most important classes in Node.

Callbacks are emitted and bound to an event by using a consistent interface, which is provided by a class known as EventEmitter. In real life, EventEmitter can be likened to anything that triggers an event for anyone to listen to.

EventEmitter implementation involves the following steps:

You can implement the EventEmitter by creating an EventEmitter instance using the following code:

var eEmitter = require('events'); // events module from node
class emitter extends eEmitter {} // EventEmitter class extended
var myeEmitter = new emitter(); // EventEmitter instance 

EventEmitter makes it possible for you to add listeners to any random event. For flexibility, multiple callbacks can be added to a single event. Either addListener(event, listener) or on(event, listener) can be used to add a listener because they perform similar functions. You can use the following code block to add listeners:

var emitter = new MyClass();
emitter.on('event', function(arg1) { … });

You can also use the following code block to add listeners:

emitter.EventEmitter.addListener(event, function(arg1) { … });

To trigger an event, you can use emit(event, arg1), as can be seen here:

EventEmitter.emit(event, arg1, arg2……)

The .emit function takes an unlimited number of arguments and passes them on to the callback(s) associated with the event.

By putting all of this code together, we have the following snippet:

var eEmitter = require('events');
class emitter extends eEmitter { }
var myEemitter = new emitter();
myEemitter.on('event', () => {
    console.log('Hey, an event just occurred!');
});
myEemitter.emit('event'); 

We can also remove a listener from an event by using the removeListener(event, listener) or removeAllListeners(event) functions. This can be done using the following code:

EventEmitter. removeAllListeners (event, arg1, arg2……)

Alternatively, you can use the following code:

EventEmitter. removeListener(event, listener) 

EventEmitter works synchronously; therefore, listeners are called in the order in which they are registered to ensure proper sequencing of events. This also helps avoid race conditions or logic errors. The setImmediate() or process.nextTick() methods make it possible for a listener to switch to an asynchronous mode.

For example, let's say we wanted an output such as "Mr. Pick Piper". We could use the following code:

console.log ("Mr.");console.log("Pick");console.log("Piper");

However, if we want an output such as "Mr. Piper Pick" using the preceding snippet, then we would introduce an event sequencing function, setImmediate(), to help the listener to switch to an asynchronous mode of operation, such as in the following code:

console.log ("Mr."); console.log("Pick");
setImmediate(function(){console.log("Piper");});

The output of the preceding function is exactly as expected:

Figure 1.7: Output using the sequencing function

eventEmitter.once(): This can be used to add a callback that is expected to just trigger once, even when an event occurs repeatedly. It is very important to keep the number of listeners to a minimum (EventEmmitter expects the setMaxListeners method to be called). If more than a maximum of 10 listeners are added to an event, a warning will be flagged.

myEmitter.emit('error', new Error('whoops!'): This emits the typical action for an error event when errors occur within an EventEmmitter instance.

This is a chunk of memory where data is stored temporarily. This non-resizable memory is designed to handle raw binary data, and the integers in it are limited to values from 0 to 255 (2^8 - 1), with each integer representing one byte.

Let's look at a real-world scenario so that we have a better understanding of what buffers are and how they work by using the following screenshot from a YouTube video:

Figure 1.8: The workings of a buffer in YouTube

When a YouTube video starts to play, and if the internet is super fast, a gray area is observed in the playing stream. This area is the buffer zone, where data is being collected and stored temporarily (usually in the RAM) to allow for continuous playing, even when the internet is disconnected. The red zone is the playing zone (data processing zone) whose length is dependent on the video's playing speed and the amount of data that has been buffered. If the browser is refreshed, the temporary storage is re-initialized and the process is restarted.

Now that we have seen a real-time application using a buffer, we will now create, read, and write to buffers in the upcoming exercises.

In this exercise, our aim is to create, read from, and write to a buffer. Before attempting this exercise, make sure you have completed all of the previous activities in this chapter. To complete this exercise, perform the following steps:

There are a few more methods for buffers, which will be covered in the following sections.

You can also create buffers using the allocUnsafe(length) method. The allocUnsafe(length) method creates an uninitialized buffer of the assigned length. When compared to the buffer.alloc() method, it is much faster, but old data in the returned buffer instance needs to be overwritten using either fill() or write().

Let's see how the allocUnsafe(length) method is being used in the following snippet:

var buf = Buffer.allocUnsafe(15);
var buf1 = Buffer.alloc(15);

console.log(buf);
console.log(buf1);

The preceding snippet yields the following output:

Figure 1.14: Output of the buffer created using allocateUnsafe(length) and alloc(length)

There are a few other methods that you need to be aware of. These are listed here:

Whenever you talk about reading data from a source and writing it to a destination, you're referring to streams (Unix pipes). A stream can be likened to an EventEmitter. There are four types of streams, and they are as follows:

A stream is said to be readable if it permits data to be read from a source, irrespective of what the source is, be it another stream, a file in a filesystem, a buffer in memory, and so on. Various data events can be emitted at various points in a stream. Thus, streams can also be referred to as instances of EventEmitters. Listening to a data event and attaching a callback are the best ways to read data from a stream. A readable stream emits a data event, and your callback executes when data is available.

Let's observe how a stream is read, a data event is emitted, and a callback is executed using the filesystem module with a readable and writable file. See the following code:

const fs = require('fs');
const file = fs.createReadStream('readTextFile.txt');
file.on('data', function(data) {
console.log('Data '+ data);
});
file.on('end', function(){
console.log('Hey!, Am Done reading Data');
});

First, we created a file named readTextFile.txt. Then, a readable stream is created using the fs.createReadStream('filename') function. It is good to know that the stream is in a static state initially, and gets to a flowing state as soon as you listen to a data event and attach a callback. Streams emit an end event when there is no more data to read.

A stream is said to be writeable if it permits data to be written to a destination, irrespective of what the destination is. It could be another stream, a file in a filesystem, a buffer in memory, and so on. Similar to readable streams, various events that are emitted at various points in writeable streams can also be referred to as instances of EventEmitter. The write() function is available on the stream instance. It makes writing data to a stream possible.

Take a look at the following snippet:

const fs = require('fs');
const readableStream = fs.createReadStream('readTextFile.txt');
const writableStream = fs.createWriteStream('writeTextFile.txt');
readableStream.on('data', function (data) {
console.log('Hey!, I am about to write what has been read from the file readTextFile.txt');
if (writableStream.write(data) === true) {
console.log('Hey!, I am done writing. Open the file writeTextFile.txt to see what has been written');
    }
else
console.log('Writing is not successful');
});

First, we load the filesystem using the require directive. Then, we create two files: readTextFile.txt and writeTextFile.txt. We then write some text string into the readTextFile.txt file and leave writeTextFile.txt blank. Using the filesystem createReadStream() function with the file path or directory as an argument, we create a readable stream. Thereafter, a writeable stream is created using the filesystem createWriteStream() function with the file path or directory as an argument. readableStream.on('data', callback) listens to a data event and attaches a callback, whereas writableStream.write(data) writes data to a writable stream. Once this snippet is run, you will realize that the text string read from readTextFile.txt has been written into the writeTextFile.txt file.

Duplex

Recall from the short description in the previous section that a duplex is a type of stream that allows you to read data from a source and write data to a destination. Examples of duplex streams are as follows:

Transform

This is another stream method that lets you modify or transform data while data is being read or written. In essence, transform streams are duplex streams with the aforementioned functionality. Examples of transform streams are as follows:

end(): When this method is called, a finish event is emitted by the stream and a notification is sent to the stream that you have finished writing to.

setEncoding(): Upon being called, the stream becomes encoded. This method is used in a situation where a readable stream needs to be encoded.

pipe(): When this method is called on a readable stream, you can read data from the source and write to the destination without normal flow management.

pause(): When this method is called on a flowing stream, the stream pauses, data will be kept in the buffer, and data events will not be emitted anymore. When this is called on a non-flowing stream, data events will not be emitted, but the stream will start flowing.

resume(): When this method is called on a paused stream, the stream starts to flow again.

unpipe(): When this method is called on a piped readable stream, the destination streams are removed from pipes.

Node has a module name File System (fs) that works with the file systems on a computer. This module, fs, performs several methods that can be implemented both synchronously and asynchronously, but there is there is no guaranteed ordering when using them.

Reading, creating, updating, deleting, and renaming are some of the operations that can be performed on any text file that's saved in a directory.

Reading Files

After the required directive has been invoked for the fs module, assuming that we have a string of text contained in the text file, the file can be read by calling the readFile() function, as shown here:

var fs = require('fs');
fs.readFile('readFileName.txt', function(err, data) {
if (err) throw err;
console.log('Read!');
  });

Creating Files

The methods that are highlighted here are made available by the fs module for creating new files:

fs.appendFile(): This method appends a given content to a file. The file will be created in this case, even if it does not exist. This function can also be used to update a file. Its implementation is as follows:

var text = ' Hey John, Welcome to Packt class.'
var fs = require('fs');
fs.appendFile('newTextFile.txt', 'text'
', function (err) {
  if (err) throw err;
  console.log('Saved!');
  });

Once the preceding code is run, you will see the designated string written into the file.

fs.open(): When this method is called, a given file is opened for writing, and if the file doesn't exist, an empty file will be created. The implementation is as follows:

var fs = require('fs');
fs.open('TextFile.txt', 'w', function (err, file) {
  if (err) throw err;
// let's assume the file doesn't exist
  console.log('An empty file created');
});

This method takes a second argument, w, which is referred to as a flag for "writing."

fs.writeFile(): When this method is called on a given file and content exists in it, the existing content is replaced by new content. In a situation where the file doesn't exist, a new file containing the given content will be created. This function can also be used to update the file, and the implementation is as follows:

Var fs = require('fs');
fs.writeFile('textFile.txt', 'Hello content!', function (err) {
if(err)throwerr;
console.log('Saved!');
});

Deleting Files

fs.unlink(): When this method is called on a given file in the filesystem, the file is deleted. The implementation is as follows:

var fs = require('fs');
fs.unlink('textFile.txt', function (err) {
  if (err) throw err;
  console.log('File deleted!');
});

Renaming Files

fs.rename(): This method takes in three arguments: the old name of the file, the new name of the file, and a callback. When this method is called on a given file in the filesystem, the file is renamed. The implementation is as follows:

var fs = require('fs');
fs.rename('textFile.txt', 'renamedtextFile.txt', function (err) {
  if (err) throw err;
  console.log('File Renamed!');
});

access(): This method is used to check whether a user has access to the file or directory.

close(): This method is used to close a file.

exists(): Though this method is deprecated, it's called on a file to check whether a file or folder exists.

fstat(): When this method is called on a file, it returns the status of that file.

link(): This method is used to create an additional name for a file. The name could either be the old or a new one.

mkdir(): This method is used make a new directory.

readdir(): This method is used to read the contents of a directory.

rmdir(): This method is used to remove a directory.

stat(): This method is used to return the status of a file.

truncate(): This method is used to truncate a file.

watch(): This method is used to watch for changes in a file name or directory name.

Our aim is to read and write data using the Node filesystem. To complete this exercise, the following steps have to be performed:

The previous examples and activities have explored the knowledge and implementation of different built-in modules in Node. The next activity will be a hands-on implementation of streaming data from one file to another.

You have been tasked with developing an application that streams data (copies some text) from a local file on your system into another local file that's located in a different directory. The aim is to read and write data to a stream.

Before we begin this activity, it is essential that you have completed all the previous exercises and activities. Then, use the IDE to create a project called StreamingProgram.

To complete this exercise, the following steps have to be completed: