Book Image

Driving DevOps with Value Stream Management

By : Cecil 'Gary' Rupp
Book Image

Driving DevOps with Value Stream Management

By: Cecil 'Gary' Rupp

Overview of this book

Value Stream Management (VSM) opens the door to maximizing your DevOps pipeline investments by improving flows and eliminating waste. VSM and DevOps together deliver value stream improvements across enterprises for a competitive advantage in the digital world. Driving DevOps with Value Stream Management provides a comprehensive review and analysis of industry-proven VSM methods and tools to integrate, streamline, and orchestrate activities within a DevOps-oriented value stream. You'll start with an introduction to the concepts of delivering value and understand how VSM methods and tools support improved value delivery from a Lean production perspective. The book covers the complexities of implementing modern CI/CD and DevOps pipelines and then guides you through an eight-step VSM methodology with the help of a use case showing an Agile team's efforts to install a CI/CD pipeline. Free from marketing hype or vendor bias, this book presents the current VSM tool vendors and customer use cases that showcase their products' strengths. As you advance through the book, you'll learn four approaches to implementing a DevOps pipeline and get guidance on choosing the best fit. By the end of this VSM book, you'll be ready to develop and execute a plan to streamline your software delivery pipelines and improve your organization's value stream delivery.
Table of Contents (23 chapters)
1
Section 1:Value Delivery
7
Section 2:VSM Methodology
13
Section 3:VSM Tool Vendors and Frameworks
18
Section 4:Applying VSM with DevOps

Taking a Lean-Agile view of value

Lean-Agile practices blend the values and principles outlined in the Manifesto for Agile Software Development, more commonly referred to as the Agile Manifesto, with the Lean Development practices initially developed at Toyota and further elaborated by John Krafcik (Krafcik, 1988), James Womack, and Daniel Jones (Womack, Jones; 1990, 2013), and Mary and Tom Poppendieck (Poppendieck, 2003).

Note

My previous book, Scaling Scrum Across Modern Enterprises, provides much greater detail on the subject of Lean-Agile concepts and practices. This section provides a gentle introduction, sufficient to understand the applications of Lean-Agile practices described in later sections of this book.

Understanding the values and principles of Agile

In 2001, 17 software developers came together at a resort in Utah to discuss their software development views to see if there was common ground from which they operated. Though many of the participants were competitors or ascribed to different software development methodologies, they found common ground in their values and principles. Jim Highsmith described their result as the "mushy stuff of values and culture" in software development (https://agilemanifesto.org/history.html).

The Agile Manifesto established 4 values and 12 principles that articulate an agile software development approach.

The essential elements of the Agile Manifesto are its values, which are expressed as follows:

We are uncovering better ways of developing software by doing it and helping others do it. Through this work, we have come to value:

  • Individuals and interactions over processes and tools
  • Working software over comprehensive documentation
  • Customer collaboration over contract negotiation
  • Responding to change over following a plan

That is, while there is value in the items on the right, we value the items on the left more.

Kent Beck James Grenning Robert C. Martin

Mike Beedle Jim Highsmith Steve Mellor

Arie van Bennekum Andrew Hunt Ken Schwaber

Alistair Cockburn Ron Jeffries Jeff Sutherland

Ward Cunningham Jon Kern Dave Thomas

Martin Fowler Brian Marick

One of the signatories of The Manifesto for Agile Software Development is Jim Highsmith. He built and maintains the website that includes additional details of the event and the attendees' findings. The Uniform Resource Locators (URLs) for those pages are provided here:

These software developers represented or used lightweight software development practices to avoid the pitfalls of the traditional plan-driven and linear-sequential SDLC model (a.k.a. Waterfall). Since they used competitive methodologies, they did not find common ground across their disparate SDLC practices, but instead, found common ground in the written abstraction of their shared values and principles.

In a modern context, Agile-based software development practices have the following in common:

  • Iterative and frequent development cycles.
  • Incremental releases of customer-centric value.
  • Small, autonomous, and self-contained teams that are nimble, adaptable, and able to self-organize to perform whatever relevant work comes their way (that is, we don't expect software development teams to perform marketing or sales-oriented work).
  • Frequent interactions with customers to demonstrate the new increments of value and receive their feedback.
  • Developers need the freedom to learn from experimentation, knowing they might fail, but also know to keep their failures small across brief development increments.
  • Frequent retrospectives to assess areas for improvements and implement action plans to install those improvements in the next iteration.
  • Technical excellence and good design enhance a team's abilities and agility.
  • Diversity and respect among team members are critical elements for evolving and achieving ongoing success as a functional team.

Beyond these simple principles, software development teams are free to use whichever methods and tools they prefer. The 17 participants represented at least 7 lightweight software development methodologies. Of those 7 approaches, among others, Scrum went on to become the de facto standard for implementing small team agility.

This introductory section concludes our discussion of Agile. We'll now move on to introduce the concepts of Lean Development, and in particular, its modern relevance to Agile software development.

Reviewing the fundamentals of Lean Development

As noted in an earlier section, Defining customer value, Lean Development concepts evolved initially at Toyota. Toyota was not shy about sharing their ideas across their value chain partners and later—on a larger scale—to manufacturing companies across industries via the promotion of a booklet titled The Toyota Way 2001. They were committed to helping their value chain partners, as those efforts paid dividends in improving their value streams.

In 2004, Jeffrey Liker summarized his view of The Toyota Way via 14 fundamental management principles (Liker, 2004). I also provide examples of these principles in my book, Scaling Scrum Across Modern Enterprises.

As a quick introduction, the following list contains a summary of each management principle:

  • Lean is a long-term management philosophy.
  • Focus on establishing continuous flows to surface problems quickly and eliminating waste.
  • Use a pull-oriented approach to order and materials entry based on demand.
  • Eliminate all forms of batch processes.
  • Build things right the first time, and stop production immediately when problems arise.
  • Standardize tasks and processes across your value streams.
  • Use visual controls such as Kanban boards to avoid hiding problems.
  • Use only thoroughly tested and reliable technologies selected by your people to support their processes.
  • Grow your future leaders, who already understand the work and organizational philosophies and who can teach it to others.
  • Develop exceptional teams and people.
  • Respect and help improve your extended network of partners and suppliers.
  • Practice Gemba—go and see for yourself what's going on in the organization.
  • Make decisions slowly, by consensus, and only after due diligence and consideration.
  • Become a learning organization through reflection and continually improving.

Since this book is about Lean-Agile practices to improve an organization's competitive position in a digital economy, let's move on to discuss Lean Software Development practices.

Implementing Lean Software Development practices

Mary Poppendieck and Tom Poppendieck coined the term Lean Software Development in a book they wrote in 2003 by the same name. Their book was the first widely promoted example of employing traditional Lean Development practices to software development. Besides adopting and applying 7 Lean principles to software development, Poppendieck's identified 22 Lean thinking tools that aid in applying Lean to various Agile practices.

The Lean principles include the following:

  • Eliminate waste: Remove anything from an activity that does not add value from the perspective of our customers.
  • Amplify learning: Dedicate time and resources to learning and experimentation to improve skills, technologies, and processes.
  • Decide as late as possible, to keep your options open.
  • Deliver as fast as possible, through integration and automation of SDLC and operational support processes.
  • Empower the team: Let the people doing the work make the most of their decisions, as they are most knowledgeable and closest to the work.
  • Build integrity in: Build software systems with coherent architectures, usability, and fitness for purpose that are maintainable, adaptable, and extendable.
  • See the whole: Avoid specialization of practices as specialists tend to optimize systems around their specific goals and interests.

The disciplines of Lean and Lean Software Development and the Lean tools recommended by the Poppendiecks are too broad to get into within this chapter. We revisit these topics throughout this book. But for now, let's move on to introduce the concepts of value streams, their purpose, their types, and how we define them.

Delivering value through value streams

James Womack, Daniel Jones, and Daniel Roos often get credit for defining the term value stream in their book, The Machine that Changed the World (Womack, Jones, Roos; 1990). Womack indeed discussed value streams in his later books, such as Lean Thinking (Womack; 1996, 2003) and Seeing the Whole Value Stream (Womack, Jones; 2003). However, James Martin was the first to define the concepts of value streams in detail in his book The Great Transition: Using the Seven Disciplines of Enterprise Engineering to Align People, Technology, and Strategy (Martin, 1995).

It's easy to associate value stream concepts with the concepts of value stream mapping in Lean, but that's an incorrect association. Martin briefly introduces lean manufacturing concepts in his book, but his association of value streams with lean manufacturing applies to reinventing workflows around customer value (Martin, 1995, p. 102). In contrast, value stream mapping is a visual technique to assess information and material flows. We'll talk more about value stream mapping in the following subsection.

Martin defines value streams as an "end-to-end collection of activities that create a result for a 'customer', who may be the ultimate customer or an internal "end-user" of the value stream". Martin also notes that a value stream must have a clear goal to satisfy or delight. Martin introduces value streams as an approach to reinvent business processes to support customer needs by developing products in the most "simple, direct, and narrowly focused fashion".

Martin wrote his book when business process reengineering (BPR) was a mainstream concept, and IT organizations worked with functional and cross-functional business organizations to streamline and automate critical business processes. Martin keenly observed that business processes traditionally evolved through successive changes driven by internal needs and not by customer needs. As a result, many business processes are bloated, inefficient, and anything but customer-centric.

Martin introduces his value stream concepts as an E2E redesign of business processes at the activity or task level to create better flows. Martin makes the point that too many business systems evolved to support then-existing business processes, and those underlying processes too often evolved to support dubious agendas and objectives.

A better approach is to eliminate the concept of business processes and instead have organizations reevaluate their enterprise as a collection of value streams, all of which likely need reinventing. This approach identifies what customers value and then assesses an organization's collections of activities that deliver the value (that is, value streams). The next step is to streamline the identified value streams. Only then should an organization consider developing information systems and automation to support their newly formed value streams.

In a modern Lean-Agile and DevOps environment, we can go faster, but the Lean-Agile concepts evolved after Martin's book came out. Today, value stream identification and value stream mapping are critical components of the Lean-Agile practitioner's toolkit.

Identifying value streams

Value streams are component processes of the broader and integrated business system that describe how stakeholders—either internal or external customers—receive value from an organization. A business value stream includes a series of steps or activities necessary to provide the products, services, and experiences our customers desire at a more fundamental level.

Value streams can support product development and delivery activities and any other customer-facing services. While development-oriented value streams can provide products and services directly to customers or resale partners, a more typical scenario is that they deliver products and services to operations or customer-facing value streams. Steps that do not add value represent waste in the eyes of the customer—in other words, customers do not want to pay for things that add waste without adding value.

It's easy to think our customers won't know about these extraneous activities, but in a competitive market, you will be quickly outed by your competitors who are paying attention and providing better customer-centric value for the money.

Lanning's concept of evaluating value as desirable customer experiences gives us a model to define our value streams. So, let's quickly look at the things customers commonly want from their product, service, and solution providers, outlined as follows:

  • Open, available, and easy access to knowledge:

    a. About products and services

    b. Comparative information—from the company, trusted representatives, and industry experts

    c. Product and pricing information

    d. Procurement—How and where to acquire the products and services

  • Simplified and possibly real-time order entry
  • Products developed with desired quality, features, capabilities, and costs
  • Upsell of product upgrades and services that may make their experience even better
  • Fulfillment—delivered when and how the customer prefers
  • Customer support—available on demand when needed; sufficient to answer their questions and resolve any problems they may be experiencing
  • Ongoing product maintenance and enhancements

This list is incomplete but is a good starting point for this discussion. Various organizational departments collaborate to define and create value streams, though specific departments may lead efforts in defining selected value streams—for example, product management usually takes the lead in defining target markets and the customers' relevant needs. In contrast, the marketing department defines the value streams necessary to create customer awareness.

Mapping value streams

In the previous section, you learned that James Martin refined the definition of the term value stream and that the lean concept of value stream mapping is separate. Martin used the term value stream to articulate the need to reinvent workflows around customer value.

Value stream mapping, also known as material and information flow mapping, is a Lean method to map and assess the current as-is and desired to-be future states of activities across the life cycle of a product or service. In other words, value stream maps provide a visual aid to document and evaluate the flow of work from the initial request through development and delivery until it reaches the customer.

As an interesting side note, the concept of mapping the flow of information and materials predates Lean, dating back to a book written in 1918 by Charles E. Knoeppel, titled Installing Efficiency Methods. Today, value stream mapping is more commonly associated with Lean manufacturing and Lean Software Development practices.

Toyota applied value stream mapping as a standard practice within The Toyota Way. Mike Rother and John Shook studied Toyota and then published a book on what they discovered on this topic, titled Learning to See: Value Stream Mapping to Add Value and Eliminate Muda. Later, Mary and Tom Poppendieck introduced Lean Software Development, in their book by the same name. Their book made value stream mapping a mainstream technique in the Lean community.

The following diagram shows a value stream map example of current and future states. We will not get into any further details on value stream mapping here. Instead, we'll hold off until Chapter 7, Mapping the Current State (VSM Step 4):

Figure 1.4 – Current/future value stream map

Figure 1.4 – Current/future value stream map

This section concludes the section on Lean-Agile concepts. The following section introduces VSM concepts, methods, and tools, and also introduces value stream mapping techniques and how they are used to identify the as-is status of a value stream and one or more desired to-be states that deliver value improvements. In the process, you will discover that development-oriented value streams feed operations-oriented value streams.