To understand and fully appreciate any concept, it is important to know where it came from and how it evolved.

Real-time web is not a new thing; one of the first attempts at making web real-time was the usage of Java applets. Many will remember chatting in Yahoo! chat rooms or playing chess, way back in the late '90s. Then came Flash and ActiveX plugins. This was not only for "fun" (for the consumer section), but also for use in the enterprise market. I worked for a BPM (Business Process Management) company in the early stages of my career, where they had built an ActiveX plugin for powering their dashboards and updating process information in real time. So why is it important now? Because the way in which real-time functionality is implemented and the cost involved in doing so has changed. From being a fancy feature in an application, it has become a necessity—a user demand. From being a hacked-in or technically challenging piece of the application, it is on its way to becoming a ratified standard in the form of WebSockets and Server-Sent Events (SSE). How did we get from static web to here?

The Web (and web applications), as we all know, is built over the HTTP protocol. HTTP is a request-response system, where the client sends a request for information to the server and the server responds with the requested information. In most cases, this information is the HTML or related information, like XML or JSON, to be rendered by the browser. This HTTP browser-server interaction is shown in the following figure:

HTTP browser-server interaction

In 1995, Sun and Netscape announced a partnership that saw Netscape bundle Sun's brand new Java runtime with its browser. This was the beginning of highly interactive web. Although they have since earned themselves a very bad reputation, applets were the pioneers in the field of real-time web. In the early days of real-time web, we saw applets being used everywhere, for chat, games, and even for banners.

In the same year, Netscape came up with a scripting language called JavaScript (originally LiveScript), and another small company called FutureWave Software started working on an animation software called FutureSplash Animator. Later, both of them became the cause of Java applets almost disappearing from the Web.

FutureWave was acquired by Macromedia in 1996 and they renamed FutureSplash Animator to Flash. Flash, as we all know, went on to rule the Web as the most widely available platform for creating animations, games, video players, and everything interactive, for the major part of the next decade.

In 1999, Microsoft used its iframe technology and JavaScript to update news and stock quotes on Internet Explorer's default home page (http://home.microsoft.com). In the same year, they released a proprietary ActiveX extension for IE, called XMLHTTP. This was the era when XML was the "in" thing and everyone wanted to use XML for anything they were doing. This XMLHTTP component was originally meant to load XML data in the page asynchronously, using JavaScript. It was soon adopted by Mozilla, Safari, and Opera, as XMLHttpRequest (or XHR, for short). But it was with the launch of Gmail (by Google) that the term AJAX (Asynchronous JavaScript and XML)—coined by Jesse James Garrett in an article titled Ajax: A New Approach to Web Applications—became the buzzword in web development. The following figure shows an AJAX Request:

AJAX Request

Gmail also shed light on the advantages of live updates to web pages and opened the floodgates to various hacks built over AJAX to push data from a server (or at least, giving the illusion of doing so).

Collectively, these technologies were referred to as Comet-a term introduced by Alex Russell on his blog in 2006. Comet was a play on the word Ajax, both being popular household cleaners in the US. Comet was not one single approach. It introduced multiple mechanisms to give the feeling of data being pushed from the server to the client. These included Hidden iframe, XHR polling, XHR long polling, and Script tag long polling (or, JSONP long polling).

Let us understand how these work, as they continue to remain the most commonly available mechanisms across all modern browsers.

The first and the easiest to implement is XHR polling, in which the browser keeps polling for data periodically, and the server keeps responding with an empty response unless it has data to send back to the browser. Following an event, such as receiving a mail, or creating/updating a record in the database, the server responds to the next polling request with new data. The following figure depicts this mechanism:

XHR polling

As you can see, there is a problem with this. The browser has to keep making requests to the server even when there is no data. This causes the server to get and process data even when there is nothing to deliver.

One of the solutions to this is to modify the server to piggyback the actual client requests by not only sending the data requested by the client, but also appending additional data that the server has, to send to the browser. The client needs to be modified to understand and act upon the additional incoming data. The HTTP piggybacking process is shown in the following figure:

HTTP piggybacking

As the new data is only sent when there is a client action, it causes delays in the data reaching the browser. The solution to receiving events quickly while avoiding frequent server queries is long polling.

In long polling, when the browser sends a request to the server, the server won't respond immediately if it doesn't have data to respond with, and will suspend the request. Once the event occurs, the server closes the suspended request by sending over a response to the client. As soon as the client receives the response, it sends a new request:

Long Polling

There are various ways in which long polling is implemented, such as forever iframe, multipart XHR, script tags with JSONP, and long-living XHR.

Though all these techniques work, these are hacks, bending HTTP and XHR to be able to do duplex communication, which is not what they are meant for.

With the rapid evolution of the web browsers lead by Firefox and then Chrome, the long-due upgrade to HTML, called HTML5, is being widely adopted. In HTML5, there are two new methods for pushing data from the server to the client. One is Server-Sent Events (SSE) and the other is the full duplex WebSockets.

Server-Sent Events attempts to standardize Comet-like communication across browsers. In this approach, there is a JavaScript API to create an event source, that is, a stream over which the server can send events. This is a unidirectional protocol. We will still be using the good old XHR. This is a good approach when you don't need full duplex communication; just push updates from the server to client.

The other specification which goes on to implement a full duplex communication protocol for the web applications is WebSockets. In WebSockets, the client initiates a socket connection with the server, which supports this protocol as well. The server and client will send and receive data on this socket connection.