Book Image

Continuous Delivery with Docker and Jenkins

By : Rafał Leszko
Book Image

Continuous Delivery with Docker and Jenkins

By: Rafał Leszko

Overview of this book

The combination of Docker and Jenkins improves your Continuous Delivery pipeline using fewer resources. It also helps you scale up your builds, automate tasks and speed up Jenkins performance with the benefits of Docker containerization. This book will explain the advantages of combining Jenkins and Docker to improve the continuous integration and delivery process of app development. It will start with setting up a Docker server and configuring Jenkins on it. It will then provide steps to build applications on Docker files and integrate them with Jenkins using continuous delivery processes such as continuous integration, automated acceptance testing, and configuration management. Moving on you will learn how to ensure quick application deployment with Docker containers along with scaling Jenkins using Docker Swarm. Next, you will get to know how to deploy applications using Docker images and testing them with Jenkins. By the end of the book, you will be enhancing the DevOps workflow by integrating the functionalities of Docker and Jenkins.
Table of Contents (10 chapters)

Prerequisites to Continuous Delivery

The rest of the book is dedicated to technical details on how to implement a successful Continuous Delivery pipeline. The success of the process, however, depends not only on the tools we present throughout the book. In this section, we take a holistic look at the whole process and define the Continuous Delivery requirements in three areas:

  • Your organization's structure and its impact on the development process
  • Your products and their technical details
  • Your development team and the practices you use

Organizational prerequisites

The way your organization works has a high impact on the success of introducing the Continuous Delivery process. It's a bit similar to introducing Scrum. Many organizations would like to use the Agile process, but they don't change their culture. You can't use Scrum in your development team unless the organization's structure is adjusted to that. For example, you need a product owner, stakeholders, and management that understands that no requirement changes are possible during the sprint. Otherwise, even with good will, you won't make it. The same applies to the Continuous Delivery process; it requires an adjustment of how the organization is structured. Let's have a look at three aspects: the DevOps culture, a client in the process, and business decisions.

DevOps culture

A long time ago, when software was written by individuals or microteams, there was no clear separation between the development, quality assurance, and operations. A person developed the code, tested it, and then put it into the production. If anything went wrong, the same person investigated the issue, fixed it, and redeployed to the production. The way the development is organized now changed gradually, when systems became larger and development teams grew. Then, engineers started to become specialized in one area. That made perfect sense, because specialization caused a boost in the productivity. However, the side effect was the communication overhead. It is especially visible if developers, QAs, and operations are under separate departments in the organization, sit in different buildings, or are outsourced to different countries. Such organization structure is no good for the Continuous Delivery process. We need something better, we need to adapt the so-called DevOps culture.

DevOps culture means, in a sense, coming back to the roots. A single person or a team is responsible for all three areas, as presented in the following diagram:

The reason why it's possible to move to the DevOps model without losing on the productivity is the automation. Most of the tasks related to the quality assurance and operations are moved to the automated delivery pipeline and can be therefore managed by the development team.

A DevOps team doesn't necessarily need to consist only of developers. A very common scenario in many organization's under transformation is to create teams with four developers, one QA, and one person from operations. They need, however, to work closely together (sit in one area, have stand-ups together, work on the same product).

The culture of small DevOps teams affects the software architecture. Functional requirements have to be well separated into (micro) services or modules, so that each team can take care of an independent part.

The impact of the organization's structure on the software architecture was already observed in 1967 and formulated as Conway's Law: "Any organization that designs a system (defined broadly) will produce a design whose structure is a copy of the organization's communication structure."

Client in the process

The role of a client (or a product owner) slightly changes during the Continuous Delivery adoption. Traditionally, clients are involved in defining requirements, answering questions from developers, attending demos, and taking part in the UAT phase to agree if what was built is what they had in mind.

In Continuous Delivery, there is no UAT, and a client is essential in the process of writing acceptance tests. For some clients, who already wrote their requirements in a testable manner, it is not a big shift. For the others, it means a change in a way of thinking to make requirements more technical-oriented.

In the Agile environment, some teams don't even accept user stories (requirements) without acceptance tests attached. Such techniques, even though they may sound too strict, often lead to better development productivity.

Business decisions

In most companies, the business has an impact on the release schedule. After all, the decision what features are delivered, and when, is related to different departments of the company (for example, marketing) and can be strategic for the enterprise. That is why the release scheduling has to be reapproached and discussed between the business and the development teams.

Obviously, there are techniques such as feature toggles or manual pipeline steps, which help with releasing features at the specified time. We will describe them later in the book. To be precise, the term Continuous Delivery is not the same as Continuous Deployment. The former means that each commit to the repository is automatically released to the production. Continuous Delivery is less strict and means that each commit ends up with a release candidate, so it allows the last step (release to the production) to be manual.

In the rest of the book, we will use the terms Continuous Delivery and Continuous Deployment interchangeably.

Technical and development prerequisites

From the technical side, there are a few requirements to keep in mind. We will discuss them throughout the book, so let's only mention them here without going into detail:

  • Automated build, test, package, and deploy operations: All operations need to be possible to automate. If we deal with the system that is non-automatable, for example, due to security reasons or its complexity, then it's impossible to create a fully automated delivery pipeline.
  • Quick pipeline execution: The pipeline must be executed in a timely manner, preferably in 5-15 minutes. If our pipeline execution takes hours or days, then it won't be possible to run it after every commit to the repository.
  • Quick failure recovery: A possibility of the quick rollback or system recovery is a must. Otherwise, we risk the production health due to frequent releases.
  • Zero-downtime deployment: The deployment cannot have any downtime since we release many times a day.
  • Trunk-based development: Developers must check in regularly into one master branch. Otherwise, if everyone develops in their own branches, the integration is rare and therefore the releases are rare, which is exactly the opposite of what we want to achieve.

We will write more on these prerequisites and how to address them throughout the book. Keeping that in mind, let's move to the last section of this chapter and introduce what system we plan to build in this book and what tools we will use for that purpose.