In their book Human Problem Solving, Allen Newell and Herbert Simon outlined the problem space theory. The theory states that humans solve problems by searching for a solution in the problem space. The problem space describes the initial and desired states, as well as possible intermediate states. It can also contain specific constraints and rules that define the context of the problem. In the software industry, people operating in the problem space are usually customers and users.
Each real problem demands a solution, and if we search properly in the problem space, we can outline which steps we need to take to move from the initial state to the desired state. This outline and all the details about the solution form a solution space.
The classic story of problem and solution spaces, which get completely detached from each other during the implementation, is the story of writing in space. The story goes that in the 1960s, space-exploring nations realized that the usual ballpoint pens wouldn't work in space due to the lack of gravity. NASA then spent a million dollars to develop a pen that would work in space, and the Soviets decided to use the good old pencil, which costs almost nothing.
This story is so compelling that it is still circulating, and was even used in the TV show The West Wing, with Martin Sheen playing the US president. It is so easy to believe, not only because we are used to wasteful spending by government-funded bodies, but mostly because we have seen so many examples of inefficiency and misinterpretation of real-world issues, adding enormous unnecessary complexity to their proposed solutions and solving problems that don't exist.
This story is a myth. NASA also tried using pencils but decided to get rid of them due to the production of microdust, tips breaking, and the potential flammability of wooden pencils. A private company called Fisher developed what is now known as a space pen. Later, NASA tested the pen and decided to use it. The company also got an order from the Soviet Union, and pens were sold across the world. The price was the same for everyone, $2.39 per pen.
Here you can see the other part of the problem space/solution space issue. Although the problem itself appeared to be simple, additional constraints, which we could also call non-functional requirements, or, to be more precise, operational requirements, made it more complicated than it looked at first glance.
Jumping to a solution is very easy, and since most of us have a rather rich experience of solving everyday problems, we can find solutions for many issues almost immediately. However, as Bart Barthelemy and Candace Dalmagne-Rouge suggest in their article When You're Innovating, Resist Looking for Solutions (2013, Harvard Business Review https://hbr.org/2013/09/when-youre-innovating-resist-l), thinking about solutions prevents our brain from thinking about the problem. Instead, we start going deeper into the solution that first came to our mind, adding more levels of detail and making it the most ideal solution for a given problem.
There's one more aspect to consider when searching for a solution to a given problem. There is a danger of fixating all your attention on one particular solution, which might not be the best one at all but it was the first to come to mind, based on your previous experiences, your current understanding of the problem, and other factors:
Refinement versus exploration
The exploratory approach to find and choose solutions involves quite a lot of work to try out a few different things, instead of concentrating on the iterative improvement of the original good idea. However, the answer that is found during this type of exploration will most probably be much more precise and valuable. We will discuss fixation on the first possible solution later in this chapter.