What this book covers
Part 1, Preparing to Test, covers the necessary steps before you can start writing the test plan.
Chapter 1, Making the Most of Exploratory Testing, describes how best to perform exploratory testing and when you should do it. The main goal of exploratory testing is not to find bugs but to understand a feature better in order to improve the feature specification and test plans. It also finds any blocking issues that may delay future testing.
Chapter 2, Writing Great Feature Specifications, shows how to write comprehensive and useful feature specifications. This document will be the basis for all subsequent testing, so it needs to cover all the questions that could arise. That requires input from multiple groups, so this document always needs a thorough review.
That specification review is covered in Chapter 3, How to Run Successful Specification Reviews, in which you check the specification with the developers and product owners. The developers should highlight any special cases or conditions they had to add code for, while the product owner makes decisions that could affect the user experience and the scope of the changes. Successfully running that review to get what testers need from it is your responsibility.
Armed with the specification and the experience of exploratory testing, you can start designing the test plan. Part 2, Functional Testing, of this book explains the many different types of functional testing, in which your product responds to different inputs.
Chapter 4, Test Types, Cases, and Environments, examines how best to write test cases and where you should run them. It shows how unit, integration, and system tests can work together and where they fit in the release cycle.
Chapter 5, Black-Box Functional Testing, is the most familiar area of testing, and the first one people think of. When you use the feature, does it work correctly and achieve its stated goals? Even within this type of testing, there are many ideas and suggestions for how to find common weaknesses and issues.
Chapter 6, White-Box Functional Testing, is informed by a knowledge of the underlying code and its architecture. Understanding how the code works lets you add another important set of tests.
Chapter 7, Testing of Error Cases, is devoted to invalid inputs or situations and how the application handles them. This is a large area of testing because the number of invalid inputs often massively outweighs the possible valid inputs. In addition, error cases are more likely to have defects because less time is spent thinking through their consequences than happy path scenarios.
Chapter 8, User Experience Testing, covers the unique set of considerations to ensure your application is easy to use. This is far more subjective than other areas of testing and may require judgments on, for instance, which command name is clearest, or which function will be used most and should be accessed most easily. It’s vital to get those questions right to give your users the best experience.
Chapter 9, Security Testing, lists common security vulnerabilities that should routinely be checked by your test plans, as well as different attacks you can craft to ensure the appropriate protections are in place.
Chapter 10, Maintainability, considers your application’s logging, event generation, and monitoring. If there is an issue, how easy will it be to diagnose and fix? This can be of lower importance to businesses, since it is not directly customer-facing, but is of high importance to the test team and other internal users who spend large amounts of time chasing down issues.
Those chapters conclude the functional areas of testing, where you perform a certain action and check a certain outcome. If the application under test has passed all these tests, then it will work for users under normal circumstances. Part 3, Non-Functional Testing, considers abnormal circumstances such as high load and system failures.
Chapter 11, Destructive Testsing, considers scenarios in which different parts of the system are deliberately disabled or degraded to ensure that the rest of the system behaves gracefully and can recover from issues.
Chapter 12, Load Testing, checks the behavior when your application runs at its maximum performance. While it may be able to perform individual actions correctly, is it reliable when they’re repeated many times? These tests also check that your application continues to perform well, even when some subsystems are placed under load.
Finally, Chapter 13, Stress Testing, describes tests that deliberately load the system beyond its capabilities – for example, if too many entities are configured or if the system is taken beyond its loading limits. As with destructive testing, the correct behavior is to fail gracefully and recover once the stress condition is lifted.
The Appendix contains an example test plan that puts these ideas into practice for an example feature – users signing up to a web page.
Applying this guide gives you the best chance of finding defects in your software. However, despite all the suggestions here, in my experience, you find the most interesting bugs when you go off the test plan. Trying out ideas for yourself or following up on something odd that you spot is a great way to think up test cases no one else has considered. Observation and curiosity are vital throughout testing, so always keep your eyes open.
The suggestions here aren’t a recipe to be followed but instead a guide from which you can create your own test plans. Testing is like fishing. You can’t guarantee what you’ll find, and there’ll be plenty of smaller catches among the big issues that you’re really seeking. However, by looking in the most promising places and using the best techniques, you give yourself the greatest chance of success. Happy bug hunting!
