Introducing DevOps in IT delivery

This book will focus on implementing and scaling DevOps in large enterprises. Before we get into the specific challenges of an enterprise, we need to have a common understanding of DevOps.

Somewhere, businesses and their leaders must have thought that it was a good idea to put developers and operators into one team. In essence, DevOps is the development and operations stages working as one team, on the same product and managing it. You build it, you run it.

DevOps has gained a lot of momentum over the past decade, especially in enterprises. But implementing DevOps turned out to be quite difficult. The reason for this is that enterprises are not organized in a structure that works for DevOps. From the last century onward, most enterprises outsourced a lot of their IT. Most of the IT muscles of a major enterprise are therefore still with system integrators and software houses. DevOps becomes more difficult when development is done by a software house and operations is outsourced to a system integrator.

DevOps starts with the business. By bringing teams together into a development and operations environment that traditionally work in silos, an enterprise can speed up development and release new products and services. The rationale behind this is that less time is needed to do handovers between development and operations. Also, by removing the barrier between development and operations, the quality of products will improve since DevOps includes quality assurance, testing, and security. Customer feedback is continuously evaluated and included in new iterations of the product.

The benefits of DevOps are as follows:

• It brings business, development, and operations together, without silos.
• Enterprises can respond faster to demands from the market because they're absorbing continuous feedback.
• Products are continuously improved and upgraded with new features, instead of planning for major next releases.
• Through automation in DevOps pipelines, enterprises can reduce costs in terms of both development and operations and, at the same time, improve the quality of their products.

It starts with the business and thus the starting point is the enterprise architecture. This is where the business goals are set and we define how these goals will be met. IT delivery is key to meeting these goals. In large enterprises, the architecture also defines the IT delivery processes and the demarcation between these processes. We will look at IT delivery and its processes in more detail in the next section.

Understanding IT delivery in enterprises

As we mentioned at the beginning of this section, large enterprises typically have an operating model that is based on outsourcing. This makes implementing DevOps more complicated. The enterprise architect will have to have a very clear view of the demarcation between the different processes and who's responsible for fulfilling these processes. Who is responsible for what, when, and why? The next question is, how does it map to DevOps?

First, we need to understand what the main processes are in IT delivery. These processes are as follows:

• Business demand: A business needs to understand what the requirements are for a product that it delivers. These requirements are set by the people who will use the product. Customers will demand a product that meets a specific functionality and quality. The architecture must focus on delivering an end product that satisfies the needs of the customers of an enterprise. IT delivery is a crucial part of delivering an end-product. In DevOps, an assigned product owner makes sure that the product meets the requirements. The product owner will have to work closely with the enterprise architect. In the Creating a reference architecture section, we will learn that the enterprise architecture and DevOps are complementary.
• Business planning: Once the demand is clear, the product needs to be scoped. In DevOps, product teams typically start with a Minimum Viable Product (MVP), a first iteration of the product that does meet the requirements of the customer. When designing the MVP, processes need to be able to support the development and operations of that product. Hence, business planning also involves quality management and testing, two major components of IT delivery. This needs to be reflected in the architecture.
• Development: In DevOps, the product team will work with user stories. A team must break down the product into components that can be defined as deliverables. For this, we must have a clear definition of the user story. A user story always has the same format: As a [function of the user] I want to [desire of the user] so that I [description of the benefits a user will get if the function has been delivered and the goal is achieved]. The key of any user story is its acceptance criteria, or the Definition of Done (DoD). When is the product really finished and does it meet the goals that have been set? In Chapter 3, Architecting for DevOps Quality, you will learn more about the DoD.

One important remark that must be made is that when we refer to a product, we are talking about a product that is code-based.

Tip

There's one major movement in IT delivery: everything in IT is shifting to code. It's one of the main principles of The Modern DevOps Manifesto: Everything is code. It applies to applications, but also to infrastructure components such as network devices, servers, and storage devices. Therefore, DevOps not only includes software development and operations for applications, but also for infrastructure with Infrastructure as Code and Configuration as Code. Public clouds such as AWS, Azure, and Google Cloud Platform play a significant role in these developments.

In other words, the team is developing code: application code, Infrastructure as Code, and also test code. A developer will work on a specific piece of code that has been defined in the product backlog. The whole end product – for instance, an application – has been broken down into product backlog items (PBIs), where each developer will work on a PBI. As soon as a piece of code is ready, it needs to be tested on itself, but also as a component of the end product. Due to this, in development, code needs to be merged. This merging process is triggered by a pull request, where the developer requests to have the code merged and joined to the end product, thus fulfilling the user story. This is done using pipelines.

In Chapter 2, Managing DevOps from Architecture, we will discuss setting up and managing pipelines, both for application development and for infrastructure.

We can divide the whole DevOps cycle into two major phases called deployment and operations, as follows:

• Deployment: In this stage, the code is tested and validated so that it matches the user story. It will now be deployed to the production state. Testing and releasing to production is a process that, ideally, is automated in the pipeline, as is integration. Before the code is actually pushed to production, it also needs to be merged with configuration. Think of security packages that need to be applied to components that run in production. In the test and quality process, the full package – application code and infrastructure components – needs to be validated as "ready for production". The result should be a "live" product. If, when you're performing testing and validation, bugs, flaws, or violations of security rules are discovered, the product will be sent back to an earlier stage in the development process.
• Operations: After deployment, the live product needs to be operated on. For this, enterprises work according to IT Service Management (ITSM) principles. The fact that operators are in the same team as developers doesn't mean that the ITSM processes are not valid anymore. An example is when incidents occur and the incident management process must be triggered. In operations, we distinguish between the following main processes:

  a) Request fulfillment

  b) Incident management

  c) Problem management (postmortem)

  d) Configuration management

  e) Change management

But DevOps adds something to this; that is, continuous integration and continuous delivery (CI/CD):

• Continuous integration (CI): CI is built on the principle of a shared repository, where code is frequently updated and shared across teams that work in the cloud environments. CI allows developers to work together on the same code at the same time. The changes in the code are directly integrated and ready to be fully tested in different test environments.
• Continuous delivery (CD): This is the automated transfer of software to test environments. The ultimate goal of CD is to bring software to production in a fully automated way. Various tests are performed automatically. After deployment, developers immediately receive feedback on the functionality of their code.

CI/CD requires a feedback loop to make it continuous. It needs feedback about the delivered products and services. This is then looped back to the developers and from there, new iterations are planned to improve the product or service.

This works well if an enterprise controls the full cycle, but large enterprises have outsourced a vast number of activities to other companies. The rationale behind this sourcing strategy is typically because a certain activity is not perceived as a core activity, and it can be done more cost-effectively by a company that specializes in such activities.

However, enterprises have gone through a massive change over the last decade. IT has become more and more important and, in some cases, has become a core activity. Banks are a good example. Banks are IT companies nowadays, and the output of their IT delivery is financial products. Due to customer demands, releases of these products with new features have become more frequent, with up to several releases per day. The consequence of this is a major shift in IT delivery itself.

The next few sections will discuss how IT delivery works in sourcing models and how it impacts successfully implementing DevOps.

IT delivery in sourcing models

In this section, we will look at the sourcing model in large enterprises. This can be quite complicated, but if we learn to think in terms of sourcing tiers, it becomes more tangible and comprehensible. This is the target enterprise model, as shown in the following diagram:

Figure 1.1 – Target enterprise model

Using this model, we can break down IT delivery into three tiers:

• Tier 1: Strategic level. This is the tier for enterprise governance. The enterprise defines the strategic business goals that are translated in the enterprise architecture. The overall architecture principles are the outcome of the enterprise architecture and drive the IT architecture, including DevOps. We will discuss this further in the Creating the reference architecture section.
• Tier 2: Tactical level. This the tier where the IT architecture is developed, including DevOps. It's also the tier where service-level agreements (SLAs) and key performance indicators (KPIs) are defined to measure the outcomes of IT delivery. You will learn more about this in the Understanding SLAs and KPIs in DevOps section.
• Tier 3: Operational or services level. At this level, the components of the architecture are detailed, including the interfaces to the various suppliers and service providers. The agreements that are defined in tier 2 must be adopted at this level so that all involved developers and operators work in the same way, with the same tools and with the same understanding of the goals. In the Understanding DevOps components section, we will learn more about this.

In practice, we see service providers also acting on tier 2; for instance, if they are involved in larger programs spanning multiple products. Tier 2 then becomes the orchestration level, where a provider is responsible for aligning different streams in the lower tier. The key takeaway is that tier 1 should always be the enterprise level, where the overall governance and architecture is defined.

In this section, we learned that a lot of enterprises have outsourced larger parts of their IT and that this can complicate the process of implementing DevOps. We learned that the strategy for the entire enterprise is at tier 1, the highest tier. DevOps sits on the lowest tiers, where projects are actually executed. This is the tier where the enterprise interfaces with sourcing companies. However, this only works if we have a clear view of the architecture. We will discuss this in the next section.