Types of testing

There are several levels of testing, but this book focuses on just one – system testing. This is the hardest part of testing because it covers the whole system working together, end to end. Because it is so difficult, it requires the thought and planning described here. That does not mean it is the only kind of testing. You should supplement all the system testing here with testing at other levels since they each have different strengths.

At the lowest level is unit testing, also known as component testing, where scripts exercise individual functions or modules. This form of testing is often performed by developers, as they know what to expect from their code and are best placed to write test harnesses to drive it. Unit tests can thoroughly test individual actions, but they can’t find issues that arise when functions are used together.

To find those issues, you need integration testing. This approach involves running entire modules isolated from the rest of the system, such as user interfaces or databases. At this stage, the testing becomes much more complex. Because far more can go wrong, it allows you to find another class of issues. While both function A and function B may work well on their own, incorrect assumptions or implementations might mean they do not work together. Integration testing finds that class of bugs. It can be carried out by either developers or dedicated testers, although it requires a significant amount of automation and is unsuitable for manual testing. Using modules or clients independently of the rest of the system needs test harnesses to stand in for areas that are not being run.

Beyond integration testing is system testing, the focus of this book. Once the whole system is working together, with all modules integrated with the others, then the final class of bugs is revealed. These issues only occur when running the full system in a realistic environment, such as loading and problems between modules. On the plus side, system testing can find all the realistic bugs that will affect users. On the downside, it is difficult to translate from system behavior down to the individual functions that are failing, and systems can be complex to test fully. This book aims to give techniques and ideas to successfully navigate that challenge.

The naming convention used here follows the International Software Testing Qualification Board (ISTQB) syllabus, which refers to component, integration, and system testing. There is some overlap with the topics covered there, although this book takes a far more hands-on, practical approach rather than focusing on the theory. By selecting the most useful pieces of the ISTQB syllabus, and expanding their practical aspects, this book will show you how to thoroughly test software features.