The DevOps lifecycle - it's all about "continuous"

An automated build helps us create an application build using build automation tools such as Apache Ant and Apache Maven. An automated build process includes the following activities:

Maven and Ant automate the build process and make it simple, repeatable, and less error prone as it is a create-once-run-multiple-times concept. Build automation is the base of any kind of automation in the application delivery pipeline:

Build automation is essential for continuous integration and the rest of the automation is effective only if the build process is automated. All CI servers, such as Jenkins, Atlassian, and Bamboo use build files for continuous integration and creating their application-delivery pipeline.

What is continuous integration? In simple words, CI is a software engineering practice where each check-in made by a developer is verified by either of the following:

This step is followed by executing a unit test against the latest changes available in the source code repository:

The main benefit of continuous integration is quick feedback based on the result of build execution. If it is successful, all is well; else, assign responsibility to the developer whose commit has broken the build, notify all stakeholders, and fix the issue.

So why is CI needed? Because it makes things simple and helps us identify bugs or errors in the code at a very early stage of development, when it is relatively easy to fix them. Just imagine if the same scenario takes place after a long duration and there are too many dependencies and complexities we need to manage. In the early stages, it is far easier to cure and fix issues; consider health issues as an analogy, and things will be clearer in this context.

Continuous integration is a development practice that requires developers to integrate code into a shared repository several times a day. Each check-in is then verified by an automated build, allowing teams to detect problems early.

CI is a significant part and in fact a base for the release-management strategy of any organization that wants to develop a DevOps culture.

Following are immediate benefits of CI:

Jenkins, Apache Continuum, Buildbot, GitLabCI, and so on are some examples of open source CI tools. AnthillPro, Atlassian Bamboo, TeamCity, Team Foundation Server, and so on are some examples of commercial CI tools.

Cloud computing is regarded as a groundbreaking innovation of recent years. It is reshaping the technology landscape. With breakthroughs made in appropriate service and business models, cloud computing has expanded to its role as a backbone for IT services. Based on experience, organizations improved from dedicated servers to consolidation and then to virtualization and cloud computing:

It has opened various opportunities in terms of the availability of application—deployment environments, considering three service models and four deployment models as shown in the following diagram:

There are four cloud deployment models, each addressing specific requirements:

Cloud computing is pivotal if we want to achieve our goals of automation to inculcate DevOps culture in any organization. Infrastructure can be treated similar to code while creating resources, configuring them, and managing resources using configuration-management tools. Cloud resources play an essential role in the successful adoption of DevOps culture. Elastic, scalable, and pay-as-you-go resource consumption enables organizations to use the same type of cloud resources in different environments. The major problems in all the environments are inconsistency and limited capacity. Cloud computing solves this problem as well as those of economic benefits.

Configuration management(CM) manages changes in the system or, to be more specific, the server runtime environment. Let's consider an example where we need to manage multiple servers with same kind of configuration. For example, we need to install Tomcat on each server. What if we need to change the port on all servers or update some packages or provide rights to some users? Any kind of modification in this scenario is a manual and, if so, error-prone process. As the same configuration is being used for all the servers, automation can be useful here. Automating installation and modification in the server runtime environment or permissions brings servers up to spec effectively.

CM is also about keeping track or versions of details related to the state of specific nodes or servers. It is a far better situation when we nee and update themselves. A centralized change can trigger this, or nodes can communicate with the CM server about whether they need to update themselves. CM tools make this process efficient when only changed behavior is updated, and the entire installation and modification isn't applied again to the server nodes.

There are many popular configuration management tools in the market, such as Chef, Puppet, Ansible, and Salt. Each tool is different in the way it works, but the characteristics and end goal are the same: to bring standardized behavior to the state changes of specific nodes without any errors.

Continuous delivery and continuous deployment are used interchangeably more often than not. However, there is a small difference between them. Continuous delivery is a process of deploying an application in any environment in an automated fashion and providing continuous feedback to improve its quality. Continuous deployment, on the other hand, is all about deploying an application with the latest changes to the production environment. In other words, we can say that continuous deployment implies continuous delivery, but the converse isn't true:

Continuous delivery is significant because of the incremental releases after short spans of implementation, or sprint in agile terms. To deploy a feature-ready application from development to testing may include multiple iterations in a sprint due to changes in the requirements or interpretation. However, at the end of a sprint, the final, feature-ready application is deployed to the production environment. Like we discussed about having multiple deployments in a testing environment even for a short span of time, it is advisable to automate such a thing. Scripts to create infrastructure and runtime environments for all environments are useful. It is easier to provision resources in such environments.

For example, to deploy an application in Microsoft Azure, we need the following resources:

Then, we need to follow these steps:

In this scenario, we may consider to use a configuration file for each environment with respect to naming conventions and paths. However, we need similar types of resources in each environment. It is possible that the configuration of resources changes according to the environment, but that can be managed in a configuration file for each environment. Automation scripts can use configuration files based on the environment and create resources and deploy an application into it. Hence, repetitive steps can be easily managed by an automated approach, and this is helpful both in continuous delivery and continuous deployment.

Continuous monitoring is a backbone of end-to-end delivery pipeline, and open source monitoring tools are like toppings on an ice cream scoop. It is desirable to monitor at almost every stage in order to have transparency about all the processes, as shown in the following diagram. It also helps us troubleshoot quickly.

Monitoring should be a well thought-out implementation of a plan and it should a part of each of the component mentioned in the following diagram. Consider monitoring practices for continuous integration to continuous delivery/deployment:

There is a likely scenario where end-to-end deployment is implemented in an automated fashion but issues arise due to coding problems, query-related problems, infrastructure related issues, and so on. We can consider different types of monitoring, as shown in the following diagram:

However, there is normally a tendency to monitor only infrastructure resources. The question one must ask is whether it is enough or whether we must focus on other types of monitoring as well. To answer this question, we must have a monitoring strategy in place in the planning stage itself. It is always better to identify stakeholders, monitoring aspects, and so on based on the culture and experience of an organization.

Continuous feedback is the last important component in the DevOps culture and provides a means of improvement and innovation. Feedback always provides improvement if it comes from stakeholders who know what they need and how the outcome should be. Feedback from the customer after deployment activities can serve as inputs to developers for improvement, as shown in the following diagram, and its correct integration will make the customer happy:

Here, we are considering a situation where a feature implementation is provided to the stakeholders and they provide their feedback. In the waterfall model, the feedback cycle is very long and hence developers may not be aware about whether the end product is what the customer asked for or whether the interpretation of what needs to be delivered was changed somewhere. In agile or DevOps culture, a shorter feedback cycle makes a major difference as stakeholders can actually see the result of a small implementation phase, and hence, the outcome is verified multiple times. If customers are not satisfied, then feedback is available at a stage where it is not very tedious to change things. In the waterfall model, this would've been a disaster as feedback used to be received very late. With time and dependencies, the complexity increases and changes in such situations takes a long time. In addition to this, no one remembers what they wrote 2 months back. Hence, a faster feedback cycle improves the overall process and connects endpoints as well as finding patterns in mistakes, learning lessons, and using improved patterns. However, continuous feedback not only improves the technical aspects of implementation but also provides a way to assess current features and whether they fit into the overall scenario or there is still room for improvement. It is important to realize that continuous feedback plays a significant role in making customers happy by providing an improved experience.