In the previous section, we were reminded of the progression of building data centers to consuming cloud resources. We can also relate this shift to how we provision, deploy, and scale our compute resources. Let's have a look at how our thinking has moved from deploying physical servers to scaling our applications and to scaling at an application function level:

• Physical servers: Scale at the physical server layer
• Virtual machines: Density efficiencies achieved by virtualizing the hardware
• Containers: Better density with faster start times
• Functions: Best density by abstracting the runtime

When designing for scale in a physical server world, we needed to predict when our peak load may exceed our current capacity. This is the point at which one or more of our servers are fully utilized and become a bottleneck for the application or service. If we had enough foresight, we could order a new server kit and get it racked and bring it into service before it's needed. Unfortunately, this isn't how internet-scale works.

The lead time and work behind getting that new gear into service could include the following processes:

• Securing a capital expenditure budget from finance or the Project Management Office (PMO)
• Procuring hardware from a vendor
• Capacity planning to confirm that there is space for the new hardware
• Updates to the colocation agreement with the hosting provider
• Scheduling engineers to travel to the site
• Licensing considerations for new software being installed

The unit of scale here is an entire server, and, as you can see from the preceding list, there is a considerable list of work ahead of you before you make the decision to scale. Compounding this problem is the fact that once you have scaled, there's no scaling back. The new hardware will live in service for years and now your baseline operational costs have increased for the duration.

The drawback of scaling by physical nodes is that we always have to spec out the gear for a peak load. This means that, during low load times, the server isn't doing a lot of work and we have spare capacity.

If we could run more workloads on this single server, we could achieve more efficient use of the hardware. This is what we mean when we talk about density – the number of workloads that we can cram into a server, virtual machine, or operating system. Here we introduce the hypervisor. A hypervisor is a layer of software that abstracts the server's operating system and applications from the underlying hardware.

By running a hypervisor on the host server, we can share hardware resources between more than one virtual operating system running simultaneously on the host. We call these guest machines or, more commonly, virtual machines (VMs). Each VM can operate independently from the other, allowing multiple tenants to use the same physical host.

The following is a diagram showing how the layers can be visualized. A hypervisor sits between the host operating system and the virtual machines and allows a layer of translation so that the guest operating systems can communicate with the underlying hardware:

Now, our unit of scale is the virtual machine. Virtual machines are configured and deployed in minutes using the hypervisor management software or through the APIs that are provided. The life cycle of a VM is typically weeks or months, though it can sometimes be years.

With virtual machines, we still have to run and maintain entire operating systems. The next evolution of compute utilizes the hardware more efficiently.

Of course, a virtual machine can still sit under-utilized or even idle when the demand is low. If you've ever worked in a role managing a fleet of servers, you would have noticed the dramatic under-utilization of compute during low demand periods. Ideally, we don't want the overhead of deploying and configuring a full operating system every time we need to run a workload.

To achieve this, we can make use of containerization technologies such as Docker. Docker allows applications running on an operating system to be isolated from each other by running inside a container. Each container holds only the application and associated dependencies (binaries and libraries):

For every operating system running on a physical server, we can run many container workloads in that same OS. Containers are more efficient to run, partly because they share the resources of a single operating system. Because each container doesn't have to run its own full operating system environment, you can achieve a better density of containers over virtual machines. This means we can make better use of the hardware resources that are available.

It may take minutes to deploy and configure a VM, but it takes seconds to initialize a container. This makes it much more efficient to be able to add capacity during a scale event, which is done by spinning up more containers. It also means you can achieve better granularity of scale, so you're not deploying more capacity than you need at the time. Our unit of scale here is down to the application level. Typically, the life cycle of a container runs for minutes or hours. A request or system may only use part of an application to achieve a result.

The next evolution of compute breaks up the applications into functions so that we can achieve further efficiencies of infrastructure use.

Finally, the next evolutionary stage in our units of compute is functions. While containerization technologies such as Docker are commonplace in cloud-native applications, we can take another step further by dividing our applications into discrete functions. Function as a Service (FaaS) is another level of abstraction and is where we get to choose the runtime to execute our code on, some execution parameters such as memory allocation, and that's about it. The hardware, operating system, and runtime are all managed for you.

Martin Fowler at https://martinfowler.com/articles/serverless.html says, "Fundamentally, FaaS is about running backend code without managing your own server systems or your own long-lived server applications.":

There are many reasons and benefits behind running serverless compute, as follows:

• You don't pay for idle resources. This means you only pay when your code is running.
• The scale is instantaneous and continuous—you can run multiple instances of the same code in a parallel.
• A sequential batch process running on a server has a ceiling of the maximum throughput of that server's resources. With serverless, this process could be executed in parallel, saving the cost of processing time while getting the results sooner. Our unit of scale here is at the function level, so we can scale relative to the size of the workload.

One of the biggest shifts that is most pertinent to me is that serverless means developers can focus more on building the code that directly relates to outcomes for the business. Developers don't have to grind their gears understanding how to harden their operating system, how to maintain configurations across platforms, or how to tune an application server to get the most performance out of their runtimes.

The tasks of system administration and operations have also changed significantly. With the complexities of the platform abstracted away, the traditional tasks to do with making sure the underlying infrastructure is available has shifted. Instead, serverless operations are more focused on proactive tasks such as monitoring service latency or observing transactions to ensure they are being completed using meaningful data.

Functions can be deployed in milliseconds and typically live for seconds. Long-running processes may not be the best fit for a serverless workload, but we'll get to this later. Next, let's learn how our evolutions in hardware abstraction have influenced software architectures.