Software development process models

As a software architect, it's important for you to understand some of the common development processes that are currently used in most enterprises. A software development process defines how people in a team produce and deliver software. In general, this process relates to a software engineering theory, called a software development process model. From the time software development was defined as an engineering process, many process models for developing software have been proposed. Let us review the traditional software models, and then look at the agile ones that are currently common.

Reviewing traditional software development process models

Some of the models introduced in the software engineering theory are already considered traditional and quite obsolete. This book does not aim to cover all of them, but here, we will give a brief explanation of the ones that are still used in some companies – waterfall and incremental models.

Understanding the waterfall model principles

This topic may appear strange in a software architecture book from 2020, but yes, you may still find companies where the most traditional software process model remains the guideline for software development. This process executes all fundamental tasks in sequence. Any software development project consists of the following steps:

• Requirements, where a product requirement document is created, and it is the basis for the software development
• Design, where the software architecture is developed according to requirements
• Implementation, where the software is programmed
• Verification, where tests are taken in the application
• Maintenance, where the cycle starts again, after a delivery

Let us look at a diagrammatic representation of this:

Figure 1.3: The waterfall development cycle (https://en.wikipedia.org/wiki/Waterfall_model)

Often, the use of waterfall models causes problems related to delays in the delivery of a functional version of the software, and user dissatisfaction due to the distance between expectations and the final product delivered. Besides, in my experience, having application tests start only after the completion of development always feels terribly stressful.

Analyzing the incremental model

Incremental development is an approach that tries to overcome the biggest problem of the waterfall model: the user can test the solution only at the end of the project. The idea of this model is to give the users opportunities to interact with the solution as early as possible so that they can give useful feedback, which will help during the development of the software.

Figure 1.4: The incremental development cycle (https://en.wikipedia.org/wiki/Incremental_build_model)

The incremental model presented in the preceding picture was introduced as an alternative to the waterfall approach. The idea of the model is to run for each increment a set of practices related to software development (Communication, Planning, Modeling, Construction, and Deployment). Although it mitigated the problems related to the lack of communication with the customer, for big projects, fewer increments were still a problem because the increments remained too long.

When the incremental approach was used on a large scale—mainly at the end of the last century—many problems related to project bureaucracy were reported, due to the large amount of documentation required. This clunky scenario caused the rise of a very important movement in the software development industry – agile.

Understanding agile software development process models

At the beginning of this century, developing software was considered one of the most chaotic activities in engineering. The percentage of software projects that failed was incredibly high, and this fact proved the need for a different approach to deal with the flexibility required by software development projects.

In 2001, the Agile Manifesto was introduced to the world, and from that time forward various agile process models were proposed. Some of them have survived up until now and are still very common.

One of the biggest differences between agile models and traditional models is the way developers interact with the customer. The message that all agile models transmit is that the faster you deliver software to the user, the better. This idea is sometimes confusing for software developers who understand this as – let's try coding, and that's all, folks!

However, there is an important observation of the Agile Manifesto that many people do not read when they start working with agile:

Figure 1.5: Manifesto for Agile software development

A software architect always needs to remember this. Agile processes do not mean a lack of discipline. Moreover, when you use the agile process, you'll quickly understand that there is no way to develop good software without discipline. On the other hand, as a software architect, you need to understand that soft means flexibility. A software project that refuses to be flexible tends to ruin itself over time.

The 12 principles behind agile are foundational to this flexible approach:

1. Continuously delivering valuable software to satisfy the customer must be the highest priority of any developer.
2. Changing requirements needs to be understood as an opportunity to make the customer more competitive.
3. Do use a weekly timescale to deliver software.
4. A software team must be composed of business people and developers.
5. A software team needs to be trusted and should have the correct environment to get the project done.
6. The best way to communicate with a software team is face to face.
7. You can understand the greatest software team achievement as when the software is really working on production.
8. Agile is working properly when it delivers sustainable development.
9. The more you invest in techniques and good design, the more agile you are.
10. Simplicity is essential.
11. The more self-organized the teams are, the better-quality delivery you will have.
12. Software teams tend to improve their behavior from time to time, analyzing and adjusting their process.

Even 20 years after the launch of the Agile Manifesto, its importance and connection to the current needs of software teams remain intact. Certainly, there are many companies where this approach is not well accepted, but as a software architect you should understand this as an opportunity to transform practices and evolve the team with you are working.

There are many techniques and models that were presented to the software community with the agile approach. The next subtopics will discuss Lean software development, Extreme Programming, and Scrum, so that you can decide, as a software architect, which ones you might use to improve your software delivery.

Lean software development

After the Agile Manifesto, the approach of Lean software development was introduced to the community as an adaptation of a well-known movement in automobile engineering; Toyota's model for building cars. The worldwide Lean manufacturing method delivers a high level of quality even with few resources.

Mary and Tom Poppendieck listed seven Lean principles for software development, really connected to agile and to the approach of many companies of this century. I've listed them here:

1. Eliminate waste: You may consider waste to be anything that will interfere with the delivery of the real need of a customer.
2. Build quality in: An organization that wants to guarantee quality needs to promote it in processes that build code from the beginning, instead of only consider it after code is being tested.
3. Create knowledge: Companies that achieved excellence have a common pattern of generating new knowledge by disciplined experimentation, documenting it, and guaranteeing that this knowledge is spread all over the organization.
4. Defer commitment: Plan decisions to its last chance before causing damage to the project.
5. Deliver fast: The faster you deliver software, the more elimination of waste you have. Companies that compete using time frequency have significant advantages over their competitors.
6. Respect people: Giving reasonable objectives to the team, together with plans that will guide them to self-organize their routine, is a matter of respecting people that you work with.
7. Optimize the whole: A Lean company improves all the cycle of value; from the moment it receives a new requirement up to the one it delivers when the software is done.

The Lean principles cause a team or company approach to improve the quality of the features that the customer really needs. It also creates a reduction in time spent on features that will not be used by the time the software is delivered to the customer. In Lean, deciding the features that are important to the customer guides the team in delivering software that matters, and this is exactly what the Agile Manifesto intends to promote in software teams.

Extreme Programming

Just before the release of the Agile Manifesto, some of the participants who designed the document, especially Kent Beck, presented to the world the Extreme Programming (XP) methodology for developing software.

XP is based on values of simplicity, communication, feedback, respect, and courage. It was considered later as a social change in programming, according to Beck in his second book about the topic. It certainly promotes a huge change in the flow of development.

XP indicates that every team should have the simplicity to do only what it was asked for, communicating face to face daily, demonstrating the software early to get feedback, respecting the expertise of each member of the team, and having the courage to tell the truth about progress and estimates, considering the team's work as a whole.

XP also delivers a set of rules. These rules may be changed by the team if they detect something is not working properly, but it's important to always maintain the values of the methodology.

These rules are divided into planning, managing, designing, coding, and testing. Don Wells has mapped XP at the site http://www.extremeprogramming.org/. Although some of the ideas of the methodology were criticized strongly by many companies and specialists, there are many good practices that are used today:

• Writing software requirements using user stories: User stories are considered an agile approach to describe user needs, together with the acceptance tests that will be used to guarantee the correct implementation.
• Divide software into iterations and deliver small releases: The practice of iterating in software development is defended by all methodologies after waterfall. The fact of delivering faster versions decreases the risks of not achieving the customer's expectations.
• Avoid working overtime and guarantee a sustainable velocity: Although this must be one of the hardest tasks a software architect may deal with, overtime working indicates something is not working properly in the process.
• Keep things simple: While developing solutions, it is quite common to try to anticipate features that the customer would like to have. This approach increases the complexity of the development and the time to market the solution. A different approach will cause high costs, and probably a low level of features that are actually used, in the system you are developing.
• Refactoring: The approach of refactoring the code continuously is good because it enables the evolution of your software and guarantees the design improvement that will truly be necessary due to the normal technical changes of the platforms you use to develop.
• Keep the customer always available: If you follow the rule from XP, you should have an expert customer inside your team. This is certainly something that is hard to get and deal with, but the main idea of this approach is guaranteeing that the customer is involved in all parts of development. As another bonus, having the customer close to your team means they understand the difficulties and expertise the team has, enabling an increase of the trust between the parties.
• Continuous integration: This practice is one of the bases of the current DevOps approach. The less difference you have between your personal code repository and the main code repository, the better.
• Code the unit test first: A unit test is an approach where you program specific code for testing a single unit (class/method) of your project. This is discussed in a current development methodology called Test-Driven Development (TDD). The main goal here is to guarantee that every business rule has its own unit test case.
• Code must be written to agreed standards: The need of determining standards for coding is connected to the idea that no matter which developer you have working on a specific part of the project, the code must be written so that any of them will understand it.
• Pair programming: Pair programming is another difficult approach to achieve in every single minute of a software project, but the technique itself—one programmer coding and the other actively observing and offering comments, criticism, and advice—is useful in critical scenarios.
• Acceptance tests: The adoption of acceptance tests to meet user stories is a good way to guarantee that new released versions of the software do not cause damage to its current needs. An even better option is to have these acceptance tests automated.

It is worth mentioning that many of these rules are today considered vital practices in different software development methodologies, including DevOps and Scrum. We will discuss DevOps later in this book, in Chapter 20, Understanding DevOps Principles. Let's get into the Scrum model right now.

Getting into the Scrum model

Scrum is an agile model for the management of software development projects. The model comes from Lean principles and is one of the widely used approaches for developing software nowadays.

As you can see in the following figure, the basis of Scrum is that you have a flexible backlog of user requirements (Product Backlog) that needs to be discussed in each agile cycle, called a Sprint. The Sprint goal (Sprint Backlog) is determined by the Scrum Team, composed of the Product Owner, the Scrum Master, and the Development Team. The Product Owner is responsible for prioritizing what will be delivered in that Sprint. During the Sprint, this person will help the team to develop the required features. The person who leads the team in the Scrum process is called the Scrum Master. All the meetings and processes are conducted by this person.

Figure 1.6: The Scrum process

It is important to notice that the Scrum process does not discuss how the software needs to be implemented, nor which activities will be done. Again, you must remember the basis of software development, discussed at the beginning of this chapter; that means Scrum needs to be implemented together with a process model. DevOps is one of the approaches that may help you use a software development process model together with Scrum. Check Chapter 20, Understanding DevOps Principles, to understand it better.