The value of an iterative and incremental approach

One of the greatest benefits of using Scrum is that it prescribes an iterative and incremental approach to software development. This is by far the most effective and efficient approach for creating software in today's world and in the next sections, we'll explain exactly why that is the case. Let's begin by remembering how we used to develop software…

The waterfall legacy

When I first started programming, we used to build our systems in distinct, single stages: first analysis, then design, then coding, and so on. Each stage would cover everything we would need to consider in order to deliver the whole system, down to the finest detail. Once the stage was complete, we would move on to the next stage of the development lifecycle and never re-visit the previous, completed stage. This is now known as the waterfall approach because each stage was like a distinct level of a waterfall, one following the other in sequential, non-repeatable fashion, as illustrated in the following figure:

Figure 1.2 – Waterfall development

As we soon came to discover, there were some serious drawbacks to this approach:

• First, it took a long time to actually deliver software to our users. Since we had to consider every possible requirement, and design and document every possible functionality before we could start coding, it would take months or often years to progress from system inception to system deployment. By that time, a competitor would have beaten us to the punch by delivering their system first or the business need for our system would have simply passed, overtaken by circumstances and changes in the market.
• Secondly, since we were moving sequentially from stage to stage, any design flaws or false assumptions that were discovered after deployment could not be fixed without a major re-haul of our system. This took a lot of time and effort.
• Finally, if requirements were changed by the customer once we were past the design stage, we would have to start pretty much from scratch again.

In short, the waterfall approach was inflexible, risky, and time-consuming. It worked well for projects with rigid, unchanging requirements that were not affected by market conditions and weren't time-critical. However, as software applications started to become more prevalent in our lives and the market expanded and diversified, such projects started becoming rarer. Gone were the days in which consumers were happy to sit and wait for the next version of their spreadsheet application to come out from one of the two companies that produced them, or to wait for their email provider to fix a bug in their email system due to there being no real alternative.

Today, customers have plenty of choices and they value the speedy delivery of working software over dependence on monopolistic software providers. For software providers nowadays, time-to-market is an essential factor in their strategy and waterfall development is just too risky and rigid to follow. Luckily, the people who came up with Agile methodologies saw this at an early stage and almost all of the Agile methodologies that were developed, especially Scrum, follow what is known as an iterative and incremental development approach. Let's find out what that means.

Iterative and incremental software development

Iterative development means developing software in small chunks repeatedly, instead of waiting for everything to be finished and delivering a large chunk at the end. It entails breaking down the requirements that need to be implemented and implementing a few at a time. So instead of having a large, big-bang software delivery at the end of the project, we have many smaller deliveries at regular intervals. These delivery intervals are known as Iterations. In Scrum, we call an iteration a sprint.

Incremental development means that each iteration builds upon software delivered by previous iterations. So, if we implement Feature A in our first iteration (let's call this version 1 of our system), then in version 2 our users will expect to see Feature A and another feature too. Sometimes, we may have to deliver Feature A again, but this time working better or faster or having fixed a bug in it. The point is, each iteration should offer something more than the previous one. This chunk of software and functionality that each iteration adds to the system is called an Increment. In Scrum, an increment is not randomly produced but is intended to achieve a specific goal, to deliver the desired functionality or to fix a specific problem. This goal is decided at the beginning of the Sprint and is known as a sprint goal.

The following figure shows the characteristics of an iterative and incremental development approach:

Figure 1.3 – Iterative and incremental development

As shown, in an incremental and iterative development cycle, there is no separation between the development stages. So, within the same iteration, our team may be designing some feature, while coding some other feature, while testing a third one, all at the same time. This approach to development gives the developers the chance to correct any mistakes, fix any issues, and inspect and adapt to changing requirements at an early stage, which means less time and effort and less risk of failure or late delivery.

In an incremental and iterative cycle, we deliver working software at the end of each sprint. So, as illustrated, for Sprint 1 we deliver a crude version of our product that doesn't do much but outline what we try to build, with some basic functionality. At the end of the sprint, we showcase our software to the stakeholders and receive feedback. At the same time, we come together as the Scrum Team to inspect and review what we did well in the sprint and what we could improve. This gives us valuable information on how to improve the product in the next sprint, but also on how to improve our working practices.

In Sprint 2, we apply the lessons learned from Sprint 1 and deliver a much more functional version of the product with more and better features. Once again, at the end of the sprint, we receive feedback, inspect, and adapt in order to improve both our product and our workflow.

In the final sprint, we deliver the whole product, fully functional. By incorporating the feedback we received and the lessons we learned in the previous sprints, we understand the customer requirements much better and have improved our productivity and teamwork. In fact, inspection and adaptation are key pillars of Scrum (more about these in Chapter 2, Scrum Theory and Principles), so it's no surprise that the Scrum framework imposes iterative and incremental development. It's one of the many reasons why doing Scrum is so beneficial. But let's look at some other reasons for doing Scrum…