I'm not going to try to suggest that while as developers we "refactor" all the time it's the only option for major changes to a system. Rather than systematically evolve the code base over time, we could simply take the knowledge learned from writing the original code base and re-write it all on the same platform. Developers often embrace re-writing because they haven't been maintaining large blocks of the code, for various reasons. They feel that over time the changed requirements have caused them to neglect the underlying design and it has become fragile, hard to maintain, and time-consuming to change. They feel if they could just start anew they could produce a much better quality code base.

While re-writing may be a viable option and will most likely get you from point A to point B, it tends to ignore many aspects of releasing software that some developers tend to not want to think about. Yes, at the end of the re-write we might end up with a better code base and it might be just as good as having refactored it over the same amount of time; but there are many consequences to committing the software to a re-write.

Committing the software to a re-write does just that, commits the team to a re-write. Writing software means moving from less functionality to more functionality. The software is "complete" when it has the functionality that the users require. This means between point A and B the new software is essentially non-functional. At any given point of time, it can only be expected to perform a subset of the final functionality (assuming it's functional at all at that given point in time). This means the business has to accept the last released version of the software during the re-write. If the software is such a bad state as to be viewed as requiring a re-write then that usually means that last released version is in a bad state and user is not overly satisfied with it.

Re-writes take a long time. Developers can re-use the concepts learned from the original code and likely avoid some of the analysis that needs to go into developing software; but it's still a monumental task to write software from scratch. Re-writes are like all software development, the schedule hinges on how well the developers can estimate their work and how well they and their process are managed. If the re-write involves the same business analysts, the same developers, the same management, and the same requirements that the original admittedly flawed software was based upon, the estimates should be suspect. If all the same inputs that went into the flawed software go into the re-write, why should we expect better quality? After all, the existing software started out with a known set of requirements and still got to its existing state. Why would essentially performing the same tasks with the same process result in better software?

So, we know we are going to get to point B from point A, but now we know the business can't have new software until we get to point B and we now realize that we're not really sure when we're going to get to point B. The time estimates may not be scrutinized at the beginning of the re-write; but if the developers are unable to keep to the schedule they will most certainly be scrutinized when they fail to maintain the schedule. If the business has been without new features or bug fixes for an extended period of time, telling them that this timeframe is expanding indefinitely (and that is how they'll view it because they were already given a timeframe, and now it's changed) will not be welcome news.

I know what you're thinking, we can deal with the fact that the re-write means the software can't respond to market changes and new requirements for an unknown and significant amount of time by having two code streams. One code stream is the original code base and the other is the re-write. One set of developers can work on the original code base, implementing bug fixes and responding to market changes by adding features; and another set of developers can work on the rewrite. And this often happens, especially due to the business's response to the pushed timeframe. If they can't get their re-write right away, they'll need something with new features and bug fixes to keep the business afloat to pay for the re-write. If two code streams aren't created the software developers are right back to where they started, shoe-horning new features and bug fixes into the "pristine" code base. This is one of the very reasons why the developers feel the original code base had problems, the fact that the new features and bug fixes had caused them to neglect the design.

Maintaining two code streams make the developers feel like they're mitigating the risks involved with having to implement new features and bug fixes into their new design. It seems like everyone they should be happy. But, how is this really any different? Why would the business even accept the re-write if they're already getting what they want? How would these new features not make their way into the re-write? This is actually the same as having one code stream; but now we have to maintain two code streams, two teams, and manage change in two code bases and in two different ways. So, have we made the problem worse?

It's easy for developers to neglect the design of the system over time, especially if the design process is based on Big Design Up Front. With Big Design Up Front, the software development lifecycle stages are assumed to progress in a way similar to a waterfall, one stage can't continue until the previous stage has completed, for example, development can't begin until "Design" has been completed. This means that adding new behavior to an existing design means going back to the Design stage, coming up with a new design: documenting it, reviewing it, and submitting it for approval.

With a process like this, the management team quickly realizes that they can't make money if every evolution of the software requires a Big Design Up Front and they plead with developers to make small changes without Big Design Up Front to be responsive to customer and market demand despite mandating Big Design Up Front at the start of the project. Developers are almost always happy to oblige because they don't like dealing with processes, and generally like dealing with documentation even less. Going back to analyzing and writing designs isn't what they really want to do, so they're happy to be asked to effectively just write code. But, the problem is that they often fail to make sure the rest of the software can accommodate the change. They simply duct tape it to the side of the existing code (like adding a feature, making sure it passes one or two change-specific use cases, and moving on). It's after months of doing this that the code base becomes so brittle and so hard to manage that no developer wants to look at it because it means really understanding the entire code base in order to make a change that doesn't cause something else to break.

A developer faced with a process like this is almost forced to fall back on a re-write in order to implement any major additions or changes to the software because that's essentially how the lifecycle is organized and what it promotes. They're certainly not presented with a process that guides or even rewards the developer for keeping on top of the entire code base. The reward system is based almost solely on delivery of functionality. It's not the developer's fault that the lifecycle is forcing them into a corner like that. Or is it?

Many developers fall back on management's lifecycle. "I'm just doing what I'm told" or "I didn't define the process". This is avoiding ownership in their work. The developer is avoiding any ownership of their work because they haven't been delegated responsibility for part of what goes into their work the process. This is partially the fault of the developer, they need to take ownership of their work (the code) and understand what it is that only they really have control over. Getting into this situation is also the fault of management (or mismanagement) because it assumes that management only manages tasks and time estimates and leaves the details of "software development" to the software developers but ties their hands behind their backs by imposing a process by which they need to follow. Refactoring is a tool to help developers take ownership of the code especially code they didn't originally author.

Unless we have an almost perfect handle on the code base that we're working with, we're unlikely to know all the implicit functionality of the system. Implicit functionality is unintended functionality that users have come to rely upon or can come to rely upon. This functionality isn't necessarily a bug (but users can come to rely upon bugs as well); but is concrete behavior of the system that users are using. This is never documented and it is a side-effect of the way the software was written (that is, not the way it was designed). I've actually worked with end-users that complained when a bug was fixed, because they communicated amongst themselves a process that included a work around for the bug. They were annoyed that they had to change this process because they were so used to doing it a certain way (the work around had essentially become a reflex to them).

The reason a re-write may be appealing is because the developers don't have a complete handle on the code base and therefore cannot have a complete handle on the implicit behavior. A re-write will neglect these implicit behaviors and almost always draw complaints from users. While re-writing is a viable technique, the applicable circumstances are rare. If a re-write is proposed for a project you are on, be sure it is thoroughly evaluated.