Summary
Instead of talking about how to do test tooling and automation, this chapter covered the things a naive test toolsmith might discover in their first few years of work – at least, we hope they might discover them. As the saying goes, there is a difference between 10 years of experience and 1 year, repeated 10 times. Those problems include the depth of all that testing is (we tend to only automate part of the effort) and that defects tend to be unique, so repeated regression tests have only marginal value. We also discussed test maintenance as a hidden cost and how testing tools tend to either over-report or under-report defects.
Finally, we introduced a handful of patterns to, if not solve, at least decrease the pain of these problems.
While this chapter focused on exercising the software at the highest level, the next chapter will start at the lower level and move up. We call this programmer-facing, or the programmer’s approach view of testing.