Book Image

Software Test Design

By : Simon Amey
Book Image

Software Test Design

By: Simon Amey

Overview of this book

Software Test Design details best practices for testing software applications and writing comprehensive test plans. Written by an expert with over twenty years of experience in the high-tech industry, this guide will provide you with training and practical examples to improve your testing skills. Thorough testing requires a thorough understanding of the functionality under test, informed by exploratory testing and described by a detailed functional specification. This book is divided into three sections, the first of which will describe how best to complete those tasks to start testing from a solid foundation. Armed with the feature specification, functional testing verifies the visible behavior of features by identifying equivalence partitions, boundary values, and other key test conditions. This section explores techniques such as black- and white-box testing, trying error cases, finding security weaknesses, improving the user experience, and how to maintain your product in the long term. The final section describes how best to test the limits of your application. How does it behave under failure conditions and can it recover? What is the maximum load it can sustain? And how does it respond when overloaded? By the end of this book, you will know how to write detailed test plans to improve the quality of your software applications.
Table of Contents (21 chapters)
Part 1 – Preparing to Test
Part 2 – Functional Testing
Part 3 – Non-Functional Testing
Appendix – Example Feature Specification

Advantages, disadvantages, and alternatives

Exploratory testing has distinct strengths and weaknesses. It is an important part of the testing cycle, but only if combined with other forms of testing to mitigate its shortcomings. Throughout this book, we’ll see how the advantages of different forms of testing complement each other and why you need a mixture of different approaches to get great coverage of a feature. Here are a few of the advantages and disadvantages of exploratory testing:

Table 1.1 – Advantages and disadvantages of exploratory testing

Table 1.1 – Advantages and disadvantages of exploratory testing

Exploratory testing is quick and easy. So long as you have the running code, you don’t need any other prerequisites, such as documentation or testing tools. That said, there should be user stories or other guidance material that you can work from to enable the feature and improve the effectiveness of your testing. Finding bugs is simple because you are watching the product as you use it; you don’t need to check a monitoring console or automated test output for errors. You can see what the system was doing at the time because you were using it yourself.

On the downside, it can be difficult to reproduce issues. Unlike an automated test that precisely performs a documented set of actions, if you were just clicking around when something strange happened, it may not be obvious what you did to trigger the problem. I once found a bug because I rearranged the windows on my screen partway through testing – changing the size of the browser window caused the issue. It took me some time to realize that had caused the problem, instead of all the actions I had taken in the application itself.

To make it easier to find the cause of bugs, you can record your session, either simply on video, within the application, or in the web browser that you are using for your tests. That takes a little time to set up and review but can be very helpful when trying to reproduce issues.

While exploratory testing doesn’t need many prerequisites, it helps to have a description of the feature so that you know any areas of functionality that aren’t obvious from its interface.

Another requirement is that it should be carried out by an experienced tester. Because exploratory testing is not reviewed or planned, its success depends on the skill of the individual carrying it out. A tester with experience will know the areas that are likely to cause bugs and can try tests that have failed in the past. They will also know what to check to find issues. A junior tester will not reach the same level of coverage.

The skills involved in exploratory testing – the curiosity and alertness it requires – are required throughout the test process, even when running more formalized test plans. Whether or not you are officially performing exploratory testing, you should always approach manual tests with this same mindset.

The coverage provided by exploratory tests is also difficult to measure. How much of a feature have you tested with exploratory testing? In practice, you will need to perform all these tests again as part of a more rigorous test plan, so you will repeat anything you do during exploratory testing. For this reason, you should limit how long you spend on these tests. They provide valuable feedback, but only so long as you are learning from it. Comprehensive testing starts later in the process.

To measure what coverage you have achieved, you can have a debrief to talk through the testing you’ve done with the product owner and developer. They can also suggest other cases to try. See Chapter 3, How to Run Successful Specification Reviews, to learn more about reviewing the specification and the scenarios that should be tested.

The final weakness of exploratory testing is that it does not cover non-functional tests. Here, you are just checking whether the feature works at all, not that it can achieve high loads, work in many environments, or recover from failure conditions. All those tests will come later but are not a priority at this stage.

The alternatives to exploratory testing include detailed specifications and preparing user stories and UI mockups, although, in practice, exploratory testing complements all those tasks. A sufficiently detailed specification should give enough detail that you can write the test plan from it directly. However, in practice, it is much easier to finish the details of the specification when you can use the code for exploratory testing. The same is true of user stories. They are very useful for defining and refining the core functionality but don’t usually cover error cases. Those can also be easier to find in real usage. User interface mockups are also massively helpful to define how a feature will look and its options, but it is still valuable to try those for real when they are available.

It may seem strange to begin the discussion of testing with exploratory testing, which can only begin once an initial version of the feature is complete. The following section describes the place of exploratory testing in the development life cycle and shows that while it might not be the first testing task that you start, it is the first you can finish.