The evolution of distributed systems

The quality of the human mind to ask for more has been the driving force behind many innovations. In the early days of computing, a single mainframe computer executed a set of batch jobs to solve a certain mathematical problem at an academic institution. Then, large business corporations wanted to own these mainframe computers to execute certain tasks that would take a long time to complete if done by humans. With the advancements in electrical and electronics engineering, computers became smaller and instead of having one computer sequentially doing all the tasks, business owners wanted to execute multiple tasks in parallel by using multiple computers. The effects of improved technology on electronic circuits and their reduced size resulted in a reduction in costs, and more and more organizations started using computers.

Instead of getting things done through a single computer, people started using multiple computers to execute certain tasks, and these computers needed to connect to communicate and share the results of their executions to complete the overall task. This is where the term distributed systems came into use.

A distributed system is a collection of components (applications) located on different networked computers that communicate and coordinate their tasks by passing messages to one another via a network to achieve a common goal.

Distributing a workload (a task at hand) among several computers poses challenges that were not present before. Some of those challenges are as follows:

• Failure handling
• Concurrency
• Security of data
• Standardizing data
• Scalability

Let's discuss these challenges in detail so that the distributed systems that we will be designing in this book can overcome these challenges well.

Failure handling

Communication between two computers flows through a network. This can be a wired network or a wireless network. In either case, the possibility of a failure at any given time is inevitable, regardless of the advancements in the telecommunications industry. As a designer of distributed systems, we should vary the failures and take the necessary measures to handle these failures. A properly designed distributed system must be capable of the following:

• Detecting failures
• Masking failures
• Tolerating failures
• Recovery from failures
• Redundancy

We will discuss handling network and system failures using the preceding techniques in detail in the upcoming chapters.

Concurrency

When multiple computers are operating to complete a task, there can be situations where multiple computers are trying to access certain resources such as databases, file servers, and printers. But these resources may be limited in that they can only be accessed by one consumer (computer) at a given time. In such situations, distributed computer systems can fail and produce unexpected results. Hence, managing the concurrency in a distributed system is a key aspect of designing robust systems. We will be discussing techniques such as messaging (with NATS) that can be used to address this concurrency challenge in upcoming chapters.

Security of data

Distributed systems move data from one computer to another via a communication channel. These communication channels are sometimes vulnerable to various types of attacks by internal and external hackers. Hence, securing data transfers across the network is a key challenge in a distributed system. There are technologies such as Secure Socket Layer (SSL) that help improve the security of wire-level communication. It is not sufficient in a scenario where systems are exposing business data to external parties (for example, customers or partners). In such scenarios, applications should have security mechanisms to protect malicious users and systems from accessing valuable business data. Several techniques have evolved in the industry to protect application data.

Some of them are as follows:

• Firewalls and proxies to filter traffic: Security through network policies and traffic rules.
• Basic authentication with a username and password: Protect applications with credentials provided to users in the form of a username and password.
• Delegated authentication with 2-legged and 3-legged OAuth flow (OAuth2, OIDC): Allow applications to access services on behalf of the users using delegated authentication.
• Two-Factor Authentication (2FA): Additional security with two security factors such as username/password and a one-time password (OTP).
• Certificate-based authentication (system-to-system): Securing application-to-application communication without user interaction using certificates.

We will be exploring these topics in detail in the upcoming chapters.

Standardizing data

The software components that are running on different computers may use different data formats and wire-level transport mechanisms to send/receive data to/from other systems. This will become a major challenge when more and more systems are introduced to the platform with different data and transport mechanisms. Hence, adhering to a common standard makes it easier to network different systems without much work. Distributed systems designers and engineers have come up with various standards in the past, such as XML, SOAP, and REST, and those standards have helped a lot in standardizing the interactions among systems. Yet there is a considerable number of essential software systems (such as ERP and CRM) that exchange data with proprietary standards and formats. On such occasions, the distributed system needs to adopt those systems via technologies by using an adapter or an enterprise service bus that can translate the communication on behalf of such systems.

Scalability

Most systems start with one or two computers running a similar number of systems and networking, which is not a difficult task. But eventually, these systems become larger and larger and sometimes grow to hundreds or thousands of computers running a similar or a greater number of different systems.

Hence, it is essential to take the necessary action at the very early stages to address the challenge of scalability. There are various networking topologies available to design the overall communication architecture, as depicted in Figure 1.1 – Networking topologies. In most cases, architects and developers start with the simplest model of point-to-point and move into a mesh architecture or star (hub) architecture eventually.

The bus topology is another common pattern most of the distributed systems adhered to in the past, and even today, there are a significant number of systems using this architecture.

Distributed systems networking architecture

The software engineers and architects who worked on these initial distributed computing system's designs and implementations have realized that different use cases require different patterns of networking. Therefore, they came up with a set of topologies based on their experiences. These topologies helped the systems engineers to configure the networks efficiently based on the problem at hand. The following diagram depicts some of the most common topologies used in distributed systems:

Figure 1.1 – Networking topologies

These topologies helped engineers solve different types of real-world problems with distributed computing. In most cases, engineers and architects started with a couple of applications connected in a point-to-point manner. When the number of applications grows, this becomes a complicated network of point-to-point connections. These models were easy to begin with, yet they were harder to manage when the number of nodes grew beyond a certain limit. In traditional IT organizations, change is something people avoid unless it is critical or near a break-even point. This reserved mindset has made many enterprise IT systems fall into the category of either a mesh or a fully connected topology, both of which are hard to scale and manage. The following diagram shows a real-world example of how complicated an IT system would look like with this sort of topology:

Figure 1.2 – Distributed system with a mesh topology

The preceding diagram depicts an architecture where multiple applications are connected in a mesh topology that eventually became an unmanageable system. There are many such examples in real IT systems where deployments become heavily complicated, with more and more applications being introduced as a part of the business's evolution.

The era of the service-oriented architecture (SOA) and the enterprise service bus (ESB)

The IT professionals who were designing and implementing these systems realized the challenge and tried to find alternative approaches to building complex distributed systems. By doing so, they identified that a bus topology with a clear separation of responsibilities and services can solve this problem. That is where the service-oriented architecture (SOA) became popular, along with the centralized enterprise service bus (ESB).

The SOA-based approach helped IT professionals build applications (services) with well-defined interfaces that abstract the internal implementation details so that the consumers of these applications would only need to integrate through the interface. This approach reduced the tight coupling of applications, which eventually ended up in a complex mesh topology with a lot of friction for change.

The SOA-based approach allowed application developers to change their internal implementations more freely, so long as they adhered to the interface definitions. The centralized service bus (ESB) was introduced to network various applications that were present in the enterprise due to various business requirements. The following diagram depicts the enterprise architecture with the bus topology, along with an ESB in the middle acting as the bus layer:

Figure 1.3 – Distributed system with the bus topology using ESB

As depicted in the preceding diagram, this architecture worked well in most use cases, and it allowed the engineers and architects to reduce the complexity of the overall system while onboarding more and more systems that were required for business growth. One challenge with this approach was that more and more complex logic and the load were handled by the centralized ESB component, and it became a central point of failure unless you deployed that with high availability. This was inevitable with this architecture and IT professionals were aware of this challenge.

Scaling for demand

With the introduction of agile development methodologies, container-based deployments, and the popularity of cloud platforms, this ESB-based architecture looked obsolete, and people were looking for better approaches to reap the benefits of these new developments. This is the time where IT professionals identified major challenges with this approach. Some of them are as follows:

• Scaling the ESB requires scaling all the services implemented in the ESB at once.
• Managing the deployment was difficult since changing one service could impact many other services.
• The ESB approach could not work with the agile development models and container-based platforms.

Most people realized that going forward with the ESB style of networking topology for distributed systems was not capable of gaining the benefits offered by technological advancements in the computing world. This challenge was not only related to ESB, but also to many applications that were developed in a manner where more and more functionalities were built into the same application. The term monolithic application was used to describe such applications.

Microservices and containers

This was the time when a set of companies called Digital Native companies came from nowhere to rule the world of business and IT. Some popular examples are Google, Facebook, Amazon, Netflix, Twitter, and Uber. These companies became so large that they couldn't support their scale of IT demand with any of the existing models. They started innovating on the infrastructure demand as well as the application delivery demands as their primary motivations. As a result of that, two technologies evolved:

• Container-based deployments
• The microservice architecture

These two innovations go hand-in-hand to solve the problems of increased demand for the aforementioned companies. Those innovations later helped organizations of all sizes due to the many advantages they brought to the table. We will explore these topics in more detail in the upcoming chapters.

Container-based deployments

Any application that runs on a distributed system requires computing power to execute its assigned tasks. Initially, all the applications ran on a physical computer (or server) that had an operating system with the relevant runtime components (for example, JDK) included. This approach worked well until people wanted to run different operating systems on the same computer (or server). That is when virtualization platforms came into the picture and users were able to run several different operating systems on the same computer, without mixing up the programs running on each operating system. This approach was called virtual machines, or VMs.

It allowed the users to run different types of programs independent from each other on the same computer, similar to programs running on separate computers. Even though this approach provided a clear separation of programs and runtimes, it also consumed additional resources for running the operating system.

As a solution to this overuse of resources by the guest operating system and other complexities with VMs, container technology was introduced. A container is a standard unit of a software package that bundles all the required code and dependencies to run a particular application. Instead of running on top of a guest operating system, similar to VMs, containers run on the same host operating system of the computer (or server). This concept was popularized with the introduction of Docker Engine as an open source project in 2013. It leveraged the existing concepts in the Linux operating system, such as cgroups and namespaces. The major difference between container platforms such as Docker and VMs is the usage of the host operating system instead of the guest operating system. This concept is depicted in the following diagram:

Figure 1.4 – Containers versus virtual machines

The following table provides the key points of distinction between containers and VMs:

Table 1.1 – Containers versus virtual machines

So far, we've gone through the evolution of the design of distributed systems and their implementation and how that evolution paved the way to the main topic of this chapter, which is the microservice architecture. We'll try to define and understand the microservice architecture in detail in the next section.