Book Image

Continuous Delivery with Docker and Jenkins, 3rd Edition - Third Edition

By : Rafał Leszko
Book Image

Continuous Delivery with Docker and Jenkins, 3rd Edition - Third Edition

By: Rafał Leszko

Overview of this book

This updated third edition of Continuous Delivery with Docker and Jenkins will explain the advantages of combining Jenkins and Docker to improve the continuous integration and delivery process of app development. You’ll start by setting up a Docker server and configuring Jenkins on it. Next, you’ll discover steps for building applications and microservices on Dockerfiles and integrating them with Jenkins using continuous delivery processes such as continuous integration, automated acceptance testing, configuration management, and Infrastructure as Code. Moving ahead, you'll learn how to ensure quick application deployment with Docker containers, along with scaling Jenkins using Kubernetes. Later, you’ll explore how to deploy applications using Docker images and test them with Jenkins. Toward the concluding chapters, the book will focus on missing parts of the CD pipeline, such as the environments and infrastructure, application versioning, and non-functional testing. By the end of this continuous integration and continuous delivery book, you’ll have gained the skills you need to enhance the DevOps workflow by integrating the functionalities of Docker and Jenkins.
Table of Contents (16 chapters)
1
Section 1 – Setting Up the Environment
5
Section 2 – Architecting and Testing an Application
9
Section 3 – Deploying an Application

Prerequisites to CD

The rest of this book is dedicated to technical details on how to implement a successful CD pipeline. The success of this process, however, depends not only on the tools we present throughout this book. In this section, we will take a holistic look at the whole process and define the CD requirements in three areas:

  • Your organization's structure and its impact on the development process
  • Your products and their technical details
  • Your development team and the practices you adopt

Let's start with the organizational prerequisites.

Organizational prerequisites

The way your organization works has a high impact on the success of introducing the CD process. It's a bit similar to introducing Scrum. Many organizations would like to use the Agile process, but they don't change their culture. You can't use Scrum in your development team unless the organization's structure has been adjusted for that. For example, you need a product owner, stakeholders, and a management team that understands that no requirement changes are possible during the sprint. Otherwise, even with good intentions, you won't make it. The same applies to the CD process; it requires you to adjust how the organization is structured. Let's have a look at three aspects: the DevOps culture, a client in the process, and business decisions.

DevOps culture

A long time ago, when software was written by individuals or micro teams, there was no clear separation between development, quality assurance, and operations. A person developed the code, tested it, and then put it into production. If anything went wrong, the same person investigated the issue, fixed it, and redeployed it to production. The way the development process is organized changed gradually; systems became larger and development teams grew. Then, engineers started to become specialized in one area. This made perfect sense as specialization caused a boost in productivity. However, the side effect was the communication overhead. This is especially visible if developers, QAs, and operations are in separate departments in the organization, sit in different buildings, or are outsourced to different countries. This organizational structure is not good for the CD process. We need something better; we need to adopt the DevOps culture.

DevOps culture means, in a sense, going back to the roots. A single person or a team is responsible for all three areas, which are shown in the following diagram:

Figure 1.6 – DevOps culture

Figure 1.6 – DevOps culture

The reason it's possible to move to the DevOps model without losing productivity is automation. Most of the tasks that are related to QA and operations are moved to the automated delivery pipeline, so they can be managed by the development team.

Information

A DevOps team doesn't necessarily need to consist of only developers. A very common scenario in many organizations that are transforming is to create teams with four developers, one QA, and one person from operations. However, they need to work closely together (sit in one area, have stand-ups together, and work on the same product).

The culture of small DevOps teams affects the software architecture. Functional requirements have to be separated into (micro) services or modules so that each team can take care of an independent part.

Information

The impact of the organization's structure on the software architecture was observed in 1967 and formulated as Conway's law: "Any organization that designs a system (defined broadly) will produce a design whose structure is a copy of the organization's communication structure."

A client in the process

The role of a client (or a product owner) changes slightly during CD adoption. Traditionally, clients are involved in defining requirements, answering questions from developers, attending demos, and taking part in the UAT phase to determine whether what was built is what they had in mind.

In CD, there is no UAT, and a client is essential in the process of writing acceptance tests. For some clients, who have already written their requirements in a testable manner, this is not a big shift. For others, it means changing their way of thinking to make the requirements more technically oriented.

Information

In the agile environment, some teams don't even accept user stories (requirements) without acceptance tests attached. These techniques, even though they may sound too strict, often lead to better development productivity.

Business decisions

In most companies, the business has an impact on the release schedule. After all, the decision of what features are delivered, and when, is related to different departments within the company (for example, marketing) and can be strategic for the enterprise. That is why the release schedule has to be re-approached and discussed between the business and the development teams.

There are techniques, such as feature toggles or manual pipeline steps, that help with releasing features at the specified time. We will describe them later in this book. To be precise, the term continuous delivery is not the same as continuous deployment. The latter means that each commit to the repository is automatically released to production. Continuous delivery is less strict and means that each commit ends up with a release candidate, so it allows the last step (from release to production) to be manual.

Note

Throughout the remainder of this book, we will use the terms continuous delivery and continuous deployment interchangeably.

Technical and development prerequisites

From the technical side, there are a few requirements to keep in mind. We will discuss them throughout this book, so let's only mention them here without going into detail:

  • Automated build, test, package, and deploy operations: All operations need to be able to be automated. If we deal with a system that is non-automatable, for example, due to security reasons or its complexity, it is impossible to create a fully automated delivery pipeline.
  • Quick pipeline execution: The pipeline must be executed promptly, preferably in 5-15 minutes. If our pipeline execution takes hours or days, it will not be possible to run it after every commit to the repository.
  • Quick failure recovery: The possibility of a quick rollback or system recovery is necessary. Otherwise, we risk production health due to frequent releases.
  • Zero-downtime deployment: The deployment cannot have any downtime since we release it many times a day.
  • Trunk-based development: Developers must check into one main branch regularly. Otherwise, if everyone develops in their branches, integration is rare, which means that releases are rare, which is exactly the opposite of what we want to achieve.

We will learn more about these prerequisites and how to address them throughout this book. With this in mind, let's move to the last section of this chapter and introduce what system we plan to build in this book and what tools we will use for that purpose.