How does Agile play a part in DevOps?
It is common to confuse the terms Agile and DevOps. Both are used together frequently and can be used interchangeably, but they are mutually exclusive terms. Where DevOps is the practice of bringing together development and operations teams, Agile is an iterative approach that focuses on collaboration, feedback, and small rapid releases.
While both are exclusive, you will find that, by the short comparison provided previously, the goals and values of DevOps are those of Agile as well. Agile is a key part of DevOps. While Agile focuses on constant changes and making developers and development cycles more efficient, DevOps brings in the operations teams to enable continuous integration and continuous delivery.
As a project delivery framework, Agile helps address some of the disadvantages of waterfall. It would be very difficult, if not impossible, to implement DevOps using waterfall practices due to the practices of continuous integration, continuous deployment, and continuous delivery. This is one major reason why, in organizations that practice DevOps well, teams use Agile as a delivery method.
The Agile manifesto
In 2001, 17 people met in the Wasatch Mountains in Snowbird, Utah. Their aim was to discuss the future of software development. What followed was an agreement on the frustration of the current situation of software development, even if the group could not agree on how to resolve the situation.
The group agreed that organizations were too focused on planning and documenting their software development cycles, which meant organizations lost focus and sight of what was important: customer satisfaction.
Most organizations discuss corporate values such as excellence and integrity, but these values have done nothing to foster people toward a better way of working, especially software developers. This was something that needed to change. Several members of the Snowbird 17, as they came to be known, already had ideas about how to revolutionize software development and start a new era. This trip to the mountains was the group's chance to define this new era.
What came out of this trip was the Agile manifesto. At just 68 words, this short and sweet document changed software development forever. The publication of the 12 principles defined in the manifesto has arguably led to the biggest change in software development. In the two decades that have followed, these 12 principles have been embraced by individuals, teams, and organizations around the world.
Defining culture
The Agile landscape is cluttered with ideas that seem to take Agile and transform them into real-world scenarios. This isn't anything new, though; in fact, the manifesto was born out of the need to find some common ground between Scrum, Crystal Clear, Extreme Programming, and other frameworks.
One of the biggest goals of the Snowbird 17 was to see if the representatives of these frameworks could agree – they did. The agreement ended up as a set of values that define a culture.
The Agile manifesto defines the following set of values:
- Individuals and interactions over processes and tools
- Working software over comprehensive software
- Customer collaboration over contract negotiation
- Responding to change over following a plan
You can look at the full manifesto that came out of the meeting in the mountains on the Agile Manifesto website at http://Agilemanifesto.org/.
Another product of the summit was the 12 principles (https://agilemanifesto.org/principles.html), which adhere to these values. They expand on the preceding four sentences that make up the values.
These 12 principles are as follows:
- Our highest priority is to satisfy the customer through early and continuous delivery of valuable software.
- Welcome changing requirements, even late in development. Agile processes harness change for the customer's competitive advantage.
- Deliver working software frequently, from a couple of weeks to a couple of months, with a preference for a shorter timescale.
- Businesspeople and developers must work together daily, throughout the project.
- Build the project around motivated individuals. Give them the environment and support they need and trust them to get the job done.
- The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.
- Working software is the primary measure of progress.
- Agile processes promote sustainable development. The sponsors, developers, and users should be able to maintain a constant pace indefinitely.
- Continuous attention to technical excellence and good design enhances agility.
- Simplicity – the art of maximizing the amount of work not done is essential.
- The best architectures, requirements, and designs emerge from self-organizing teams.
- At regular intervals, the team reflects on how to become more effective, then tune and adjust their behavior accordingly.
Even if you had very little exposure to Agile and DevOps before reading this book, in those 12 principles, I hope you can see the connection between what we have explored so far and the principles of the Agile manifesto.
Do Agile and DevOps work together?
Agile and DevOps can sound like very different concepts. In fact, most people I speak to in the early stages of transformation are very confused by the idea of both. This confusion is also compounded as both have their own jargon and slogans. It is common for people to become frustrated with the plethora of definitions for DevOps.
Most people think that when you say Agile, you mean Scrum, and that when you say DevOps, you really mean continuous delivery. This simplification then creates tension between Agile and DevOps, to the degree that you would be forgiven for not realizing that Agile and DevOps are friends.
It was back in 2008, at the Agile Conference, that a session about Agile Infrastructure by Patrick Debois and Andrew Clay Schafer was undertaken, and the connection to DevOps was born. Patrick later coined the term DevOps, and the Agile Conference continues with a DevOps track to this day.
There is more to this than history, though. Now, let's look at the practical connections between Agile and DevOps by looking deeper than Scrum and continuous delivery.
Agile is more than Scrum
At the point when the limitations of the business or the work itself request something else, a deft group will use the basic standards of Scrum, review their practices, and adjust them to turn out to be more viable. This is especially significant when Scrum is applied external to the setting of programming advancement.
Dealing with unplanned work
In the DevOps people group, those with Agile experience recognize that Scrum is helpful for following arranged work. Some work in activities can be arranged: delivering a major framework change, moving between server farms, or performing framework overhauls. In any case, a large part of crafting tasks is spontaneous: execution spikes, framework blackouts, and traded off security. These occasions request quick reaction. There's no longer an ideal opportunity to trust that the things will be organized in excess or for the following run arranging meeting. Consequently, numerous groups that have come to grasp DevOps thinking look past Scrum to Kanban. This encourages them track the two sorts of work and causes them to comprehend the interaction between them. Then again, they embrace a cross-breed approach, frequently called Scrumban or Kanplan (Kanban with an accumulation).
From various perspectives, the way into Scrum's wide appropriation might be that it endorses no specialized practices. Scrum's lightweight administration rehearses frequently have a major effect on a group. Where there was once contending needs from different experts, there is currently a solitary arrangement of needs in the overabundance. What's more, where there was once an excessive amount of work in advancement, there is currently an arrangement that's obliged by what time has demonstrated is truly conceivable. In the mix, these can take a group higher than ever in terms of efficiency. Be that as it may, the group may wind up obliged due to the absence of specialized practices, such as coding audits, computerized acknowledgment tests, and persistent joining.
Other Agile cycles such as Extreme Programming have solid feelings about how specialized practices uphold the group's capacity to keep an economical movement and give straightforwardness and perceivability to executives and partners. Some Scrum groups resort to placing specialized undertakings in this overabundance. While that fits well for the direction of Scrum, it rapidly hits the handy issue of Product Owner inclination toward highlights. Unless the Product Owner is very specialized, she or he might not have the right stuff to assess the cost/advantage of specialized practices. That gets much harder for a Product Owner as the specialized assignments stretch into tasks to help unwavering quality, execution, and security.
What is Scrum?
Scrum is a system that assists groups with cooperating. Scrum urges groups to learn through encounters, self-coordinate while dealing with an issue, and consider their successes and misfortunes to constantly improve.
While the Scrum I'm discussing is most often utilized by programming improvement groups, its standards and exercises can be applied to a wide range of cooperation. This is one reason Scrum is so well-known. Regularly considered as a coordinated venture of the board system, Scrum depicts a bunch of gatherings, apparatuses, and jobs that work together to help groups structure and deal with their work.
Applying Scrum within your organization is no easy task and you will come up against a whole new set of terminology. Here is a short list, which is by no means exhaustive:
- Sprints
- Sprint planning
- Ceremonies
- Backlog
- Retrospective
- Standups
While Scrum is probably one of the most common frameworks in Agile, many others do exist. Kanban and Kanplan, for example, as we will discuss next, are useful for organizations that are new to Agile.
Kanban
Kanban is a well-known structure that's used to execute Agile and DevOps programming advancement. It requires continuous communication of work limits and provides a clear view of work which is planned, in progress, and completed.
Kanban works by placing work on a physical or digital board. This visualization enables you limit work in progress and maximize your efficiency, sometimes referred to as the flow of your teams.
Many people use a form of Kanban board for their everyday work. In fact, I know plenty of people who use one for everyday common tasks around the home. The board is split into various columns, and these columns define the different statuses of tasks.
Your Kanban board will also define limits around work in progress items, delivery points, and commitment points.
Kanplan
You may not have heard of the term Kanplan before. It is a mixture of methodologies, but something that may be right for your team.
Important note
When it comes to picking an Agile framework for your team, there is no silver bullet. Different methodologies in the framework have pros and cons based on many different parameters. Every team will need to determine which framework will work best for them when it comes to planning, tracking, and releasing software.
Kanplan combines features from both Scrum and Kanban. It is ideal for teams who want the ability to groom their backlog, but do not have the ability to work in sprints. It provides a great mix because teams cannot always apply the whole of Scrum, including sprints, but can quite easily work with Kanplan to start getting a better handle on their work, work in progress, their backlog, and the priority of the work in that backlog.
Mixing methodologies within organizations
There is nothing wrong at all with different teams adopting different methodologies of the Agile framework, mixing them together, and making it work for them. I've not worked with any organization who can simply lift something out of an Agile textbook and implement it in their organization.
Think of operational teams who cannot work with traditional Scrum, mostly because their work contains elements of unplanned work such as incidents. For them, Kanban works well as it puts no emphasis on planning. Think of a full DevOps teams working on a product within your organization. Here, the normal approach of Scrum would work for them as everything is within their control and they have no reliance on external teams.
Finally, think of engineering teams who want to be more Agile, but work with other teams who do not practice any level of Agile. This is a tricky situation as there is a need to be more Agile to deliver better quality but no appetite in the rest of the organization to adopt Agile methodologies. In this example, Kanplan would work well for them, giving them the ability to groom a backlog of work based on priority, then work in a Kanban-style board, which enables them to visually see work, work in progress limits, and work done.
This approach is a great starting point for teams who fit this description, and it will enable them to work toward a higher quality of work, integrating some of the technical methods of DevOps without needing the rest of the organization to follow suit.
Scaling Agile teams
Through what we've learned so far, we can see how implementing Agile in our organization can provide several benefits. However, organizations are bigger than individual teams, and you may have several teams working on one product. It is at this point that we need to understand how Agile can be scaled up to many teams within one organization. As opposed to implementing Agile at an individual team level, which is relatively easy, implementing across the organization is a real challenge.
Scaling Agile at an enterprise level requires that we adopt Agile concepts, as well as Lean-Agile at a functional level. This includes finance, procurement, HR, and sales, for example. At the enterprise level, Agile is practices being implemented across many teams and lots of engineers working in a portfolio manner.
Important note
Scaling Agile in the enterprise requires you to think functionally. So far, what we have explored is at a team level. To scale Agile, you must think of it as an organization-wide effort.
Netflix coined the phrase highly aligned, loosely coupled, and you can still see this phrase on their main jobs page. It describes a highly mature organization that uses Agile development across the whole enterprise.
Two models that are highly popular when it comes to scaling Agile at the enterprise level are the Scaled Agile Framework (SAFe) (https://www.scaledagile.com/enterprise-solutions/what-is-safe/) and the Spotify model for scaling Agile in the enterprise (https://www.atlassian.com/agile/agile-at-scale/spotify). Both are very popular, so let's look at them both in more detail.
Scaled Agile Framework
Taken directly from the SAFe website, this is the definition of SAFe:
The framework defines four core values:
- Alignment
- Built-in quality
- Transparency
- Program execution
SAFe is actually pretty broad and covers four primary areas: Agile development, lean product development, systems thinking, and DevOps. However, its core places more value on the four values described in the preceding list.
Alignment is needed to ensure you keep pace with changes that are happening fast, disruptive competitive forces, and geographically dispersed teams. Alignment is key in these scenarios since Agile teams are great, but strategy and alignment does not and cannot rest with opinions from all the Agile teams. Alignment comes from enterprise-level business objectives.
The larger the system, the higher the quality. There can be no argument as to the importance of quality, especially in large systems. Built-in quality ensures that every element in the overall solution reflects the quality that's required over the entire life cycle of development. You can think of this with Agile testing, behavior-driven development (BDD), and test-driven development (TDD).
Transparency stems from the fact that it is difficult to develop solutions. As things go wrong or do not go as planned, without a high level of transparency, the facts become obscure, and any decision-making process will not be based on actual data where the best decisions are taken. Building trust takes time, however, and transparency is a source of trust, which is provided at several levels through SAFe. Above all, none of this matters if the teams are unable to perform or deliver value on an ongoing basis. SAFe places a strong focus on working systems and the business outcomes. We know that while many organizations begin transforming with individual teams, they become frustrated as these teams then struggle to deliver more value reliably and efficiently.
Spotify model for scaling Agile
With a large number of globally distributed engineers, a key part of the success of Spotify is the company's approach to ensuring work is organized in a way that enhances agility. Throughout the journey that engineering teams at Spotify have gone through, this has been documented and shared with the rest of the world.
This model, now known as the Spotify model, is a people focused approach that focuses on autonomy for scaling Agile, with a strong focus on network and culture. Over the years, this has helped Spotify and many other organizations increase their levels of innovation and productivity by focusing on autonomy, communication, collaboration, accountability, and quality, but overall, delivery.
Important note
While commonly known as the Spotify model, it's not a framework. This is simply Spotify's view of how to scale Agile from both a cultural and technical perspective. It is one example of how to organize multiple teams in a product environment.
First introduced in 2012 (https://blog.crisp.se/wp-content/uploads/2012/11/SpotifyScaling.pdf), the model has been subject to much scrutiny from experts in the field. Upon inspection, it shows the radically simple way that Spotify has approached levels of agility. Since then, it's generated a large amount of buzz and became popular.
The overarching theme is the championing of team autonomy, and it has several ways of describing the structure of your organization. It does so by using the following terms:
- Squads
- Tribes
- Chapters
- Guilds
Squads are teams that are organized into Tribes, Chapters help subject matter experts keep in touch with each other, and Guilds are there to help people keep aligned across the whole organization, where Chapters enable you to keep together within a single Tribe.
Important note
Like any other Agile model, it's important to make sure that if you implement it within your organization, you have the feedback and transparency in place to generate and sustain a culture of trust and autonomy.
Now that we understand what role Agile has to play in DevOps, we know it's central to DevOps in so many ways and that it is vitally important if we want to succeed.