Let's try to understand what DevOps is. Is it a real, technical word? No, because DevOps is not just about technical stuff. It is also neither simply a technology nor an innovation. In simple terms, DevOps is a blend of complex terminologies. It can be considered as a concept, culture, development and operational philosophy, or a movement.
To understand DevOps, let's revisit the old days of any IT organization. Consider there are multiple environments where an application is deployed. The following sequence of events takes place when any new feature is implemented or bug fixed:
The development team writes code to implement a new feature or fix a bug. This new code is deployed to the development environment and generally tested by the development team.
The new code is deployed to the QA environment, where it is verified by the testing team.
The code is then provided to the operations team for deploying it to the production environment.
The operations team is responsible for managing and maintaining the code.
Let's list the possible issues in this approach:
The transition of the current application build from the development environment to the production environment takes weeks or months.
The priorities of the development team, QA team, and IT operations team are different in an organization and effective, and efficient co-ordination becomes a necessity for smooth operations.
The development team is focused on the latest development release, while the operations team cares about the stability of the production environment.
The development and operations teams are not aware of each other's work and work culture.
Both teams work in different types of environments; there is a possibility that the development team has resource constraints and they therefore, use a different kind of configuration. It may work on the
localhostor in the dev environmentThe operations team works on production resources and there will, therefore, be a huge gap in the configuration and deployment environments. It may not work where it needs to run – in the production environment.
Assumptions are key in such a scenario, and it is improbable that both teams will work under the same set of assumptions.
There is manual work involved in setting up the runtime environment and configuration and deployment activities. The biggest issue with the manual application–deployment process is its nonrepeatability and error-prone nature.
The development team has the executable files, configuration files, database scripts, and deployment documentation. They provide it to the operations team. All these artifacts are verified on the development environment and not in production or staging.
Each team may take a different approach for setting up the runtime environment and the configuration and deployment activities, considering resource constraints and resource availability.
In addition, the deployment process needs to be documented for future usage. Now, maintaining the documentation is a time-consuming task that requires collaboration between different stakeholders.
Both teams work separately and hence there can be a situation where both use different automation techniques.
Both teams are unaware of the challenges faced by each other and hence may not be able to visualize or understand an ideal scenario in which the application works.
While the operations team is busy in deployment activities, the development team may get another request for a feature implementation or bug fix; in such a case, if the operations team faces any issues in deployment, they may try to consult the development team, who are already occupied with the new implementation request. This results in communication gaps, and the required collaboration may not happen.
There is hardly any collaboration between the development team and the operations team. Poor collaboration causes many issues in the application's deployment to different environments, resulting in back-and-forth communication through e-mail, chat, calls, meetings, and so on, and it often ends in quick fixes.
Challenges for the development team:
The competitive market creates on-time delivery pressure.
They have to take care of production-ready code management and new feature implementation.
The release cycle is often long and hence the development team has to make assumptions before the application deployment finally takes place. In such a scenario, it takes more time to fix the issues that occurred during deployment in the staging or production environment.
Challenges for the operations team:
Resource contention: It's difficult to handle increasing resource demands
Redesigning or tweaking: This is needed to run the application in the production environment
Diagnosing and rectifying: They are supposed to diagnose and rectify issues after application deployment in isolation
Time changes everything. In the modern era, customers expect and demand extremely quick response, and we need to deliver new features continuously to stay in business. Users and customers today have rapidly changing needs; they expect 24/7 connectivity and reliability and access services over smartphones, tablets, and PCs. As software product vendors—irrespective of whether in the development and/or operations—organizations need to push updates frequently to satisfy customers' needs and stay relevant. In short, organizations are facing the following challenges:
A change in the behavior of customers or market demand affects the development process.
The waterfall model follows sequential application design process for software development. It comes with good control but lacks revisions. It is a goal based development but without any scope of revision. The waterfall model has long been used for software development:
It has its advantages, as follows:
Easy to understand
Easy to manage—the input and output of each phase is defined
Sequential process—order is maintained
Better control
However, it is only useful in scenarios where requirements are predefined and fixed. As it is a rigid model with a sequential process, we can't go back to any phase and change things. It has its share of disadvantages, as follows:
No revision
No outcome or application package until all phases are completed
Not possible to integrate feedback until all phases are completed
Not suitable for changing requirements
Not suitable for long-term and complex projects
Inefficient estimation, long time to market, and other issues led to a change in the waterfall model, resulting in the agile model. Agile development or the agile methodology is a method of building an application by empowering individuals and encouraging interactions, giving importance to working software, customer collaboration—using feedback for improvement in subsequent steps—and responding to change in an efficient manner. It emphasizes customer satisfaction through continuous delivery in small interactions for specific features in short timelines or sprints.
The following diagram illustrates the working mechanism of agile:
One of the most attractive benefits of agile development is continuous delivery in short time frames or, in agile terms, sprints. Now, it is not a one-time deployment, but multiple deployments. Why? After each sprint, a version of the application with some features is ready for showcasing. It needs to be deployed in specific environments for demonstration, and thus, deployment is no longer a one-time activity.
It is very essential from an organization's perspective to meet changing demands of customers. To make it more efficient, communication and collaboration between all cross-functional teams is essential. Many organizations have adopted the agile methodologies.
In such a case, traditional manual deployment processes work as speed barriers for incremental deployments. Hence, it is necessary to change other processes as well along with a change in the application development methodology. One key can't be used for all locks; similarly, the waterfall model is not suitable for all projects. We need to understand that agile is customer focused and feedback is vital. Changes happen based on customer feedback, and release cycles may increase. Just imagine a scenario where inputs are high but input processing is slow. Consider an example of a shoe company where one department prepares shoes and another department works on final touches and packaging. What would happen if the packaging process were slow and inefficient? Shoes would pile up in the packaging department. Now let's add a twist to this situation. What if the shoe-making department brings new machines and improves the process of making shoes? Let's say it makes the shoe-making process two to three times faster. Imagine the state of the packaging department. Similarly, cloud computing and DevOps have gained momentum, which increases the speed of delivery and improves the quality of the end product. Thus, the agile approach of application development, improvement in technology, and disruptive innovations and approaches have created a gap between development and operations teams.
DevOps attempts to fill these gaps by developing a partnership between the development and operations teams. The DevOps movement emphasizes communication, collaboration, and integration between software developers and IT operations. DevOps promotes collaboration, and collaboration is facilitated by automation and orchestration in order to improve processes. In other words, DevOps essentially extends the continuous development goals of the agile movement to continuous integration and release. DevOps is a combination of agile practices and processes leveraging the benefits of cloud solutions. Agile development and testing methodologies help us meet the goals of continuously integrating, developing, building, deploying, testing, and releasing applications. It provides a mechanism for constant feedback from different teams and stakeholders. It also provides transparency in the form of a platform for collaboration across teams, such as business analysts, developers, and testers. In short, agile and DevOps are compatible and increase each other's value.
One of the most popular sayings is that practice makes a man perfect. What if that saying were applied to a production-like environment? It is much easier to repeat the entire process as there are no last minute—surprises, and most of the issues in deployment have already been experienced and dealt with. The development team supports operational requirements such as deploy scripts, diagnostics, and load and performance testing from the beginning of the application—delivery lifecycle, and the operations team provides knowledgeable support and feedback before, during, and after deployment. The remedy is to integrate the testing, deployment, and release activities into the development process. This is done by performing all activities multiple times and making then an ongoing part of development so that by the time you are ready to release your system into production there is little to no risk, because the deployment process has already been rehearsed on many different environments in progressively more production-like environments.
A major challenge is managing the infrastructure for all environments. Virtualization and cloud environments can help you get started with this. The cloud helps us overcome this hurdle by providing flexible on-demand resources and environments. It provides distributed access across the globe and helps in the effective utilization of resources. The cloud provides a repository of software—tools that can be used on an on-demand basis. We can clone environments and reproduce required versions as and when required. The entire development, test, and production environments can be monitored and managed using the facilities provided by cloud providers. With the advent of cloud computing, it is easy to recreate every piece of infrastructure used by an application using automation. This means that operating systems, OS configuration, runtime environments and configuration, infrastructure configuration, and so forth can all be managed. In this way, it is easy to recreate the production environment exactly in an automated fashion. Thus, DevOps on cloud brings in the best-of-breed solution from both agile development and cloud solutions. It helps in providing a distributed agile environment in the cloud, leading to continuous accelerated delivery.
DevOps is effective because of new methodologies, automation tools, agile resources of cloud service providers, and other disruptive innovations, practices, and technologies. However, it is not only about tools and technology-DevOps is more about culture than tools or technology alone.
"Technology is just a tool. In terms of getting the kids working together and motivating them, the teacher is the most important." | ||
| -- Bill Gates | ||
There is an urgent need of a huge change in the way development and operations teams collaborate and communicate. Organizations need to have a change in culture and have long term business goals that include DevOps in their vision. It is important to establish the pain points and obstacles experienced by different teams or business units and use that knowledge for refining business strategy and fixing goals.
"People always fear change. People feared electricity when it was invented, didn't they? People feared coal; they feared gas-powered engines... There will always be ignorance, and ignorance leads to fear. But with time, people will come to accept their silicon masters | ||
| -- Bill Gates | ||
If we identify the common issues faced by different sections of an organization and change the strategy to bring more value, then it makes sense. It can be a stepping stone in the direction of DevOps. With old values and objectives, it is difficult to adopt any new path. It is very important to align people with the new process first. For example, a team has to understand the value of the agile methodology; else, they will resist using it. They might resist it because they are comfortable with the old process. Hence, it is important to make them realize the benefit as well as empowering them to bring about the change.
"Change is hard because people overestimate the value of what they have—and underestimate the value of what they may gain by giving that up." | ||
| -- James Belasco and Ralph Stayer | ||
Self-dependent teams bring out the best in them when they are empowered. We also need to understand that power comes with accountability and responsibility. Cross-functional teams work together and enhance quality by providing their expertise in the development process; however, it is not an isolated function. Communication and collaboration across teams makes quality way higher.
The end objective of the DevOps culture is continuous improvement. We learn from our mistakes, and it becomes experience. Experience helps us identify robust design patterns and minimize errors in processes. This leads to an enhancement of productivity, and hence, we achieve new heights with continuous innovations.
"Software innovation, like almost every other kind of innovation, requires the ability to collaborate and share ideas with other people, and to sit down and talk with customers and get their feedback and understand their needs." | ||
| -- Bill Gates | ||
