Mapping out new features
To start exploratory testing, you need three things: to be confident that the code is stable enough to test, to be running the required versions, and to have the correct configuration in place. Once they are ready, exploratory testing can begin.
It’s important to keep in mind the purpose of exploratory testing. This is not detailed testing with results you will rely on in the future. Everything you do during exploratory testing is likely to be repeated later in a more formal round of testing, so exploratory tests should be limited in time and not take too long. It is easy to waste time duplicating effort between this and later test rounds. You should only test until you have met the three goals of exploratory testing:
- To learn about the feature to plan further testing.
- To identify the tools and knowledge you need to perform further testing.
- To uncover bugs that block further testing.
Firstly, and most importantly, you can learn about the feature to prepare the feature specification. In an ideal world, you would prepare the feature specification in advance, and the implementation would match it exactly. However, during development, there are often changes; for instance, some functions might be postponed for later releases. Sometimes, later discussions and changes don’t make it to the specification, especially about the user interface. Product specifications often don’t detail error cases, security, and loading behavior. Exploratory testing is your chance to see what changes were completed and which weren’t in this release, which parts of the feature were dropped, and any that were added.
Good exploratory testing will check every screen, read every field, enter details in every input, and press every button at least once. Don’t aim to cover all possible input values or perform exhaustive testing on the functionality; just see as much as possible. Some examples of areas you should aim to cover are as follows:
- Loading every page/screen
- Entering details into every input
- Using every function
- Transitioning through every state (dialing, ringing, on-call, hanging up, for example, or signing up, awaiting verification, verified, logged in, and so on)
- Checking user-visible outputs
- Checking internal state (via logs, database contents, and more)
By touring the feature, you can find out how it works for real and bring the specification to life. It is much easier to find issues and think through consequences when you have a working feature in front of you than for the developers and product owners who could only imagine what it looked like. Make the most of that advantage.
It may be that some aspects of the feature cannot be used, for instance, if it is an API and you need to implement a client to drive it, or if you need to generate specific datasets before you can use them. This is the second aim of exploratory testing: to discover what you are missing to perform detailed testing in practice. Again, hopefully, this was clear from the initial feature specification and test planning. However, exploratory testing is a vital check that those plans can be used for real and to find any alterations you need to make.
By the end of this testing, you should know all the configuration options relevant to this feature and their effects. That will be vital to map out the dependent and independent variables for comprehensive functional testing. For instance, tracking a user’s age may just be for information and not change any behavior of this feature or product. It is written to the database and only read to simply be displayed back to the user. In that case, it is independent of other aspects of the feature. Or it may be that certain options or behaviors only appear for users of certain ages. Then, you will need to check each combination of age and those features to ensure correct behavior in each case. This is your chance to see those interactions.
The third aim of exploratory testing is to find any major issues that will block further testing. In the same way that getting to test number one is on the critical path for releasing a feature, so are any bugs so serious that they block entire sections of the test plan. For instance, if a page doesn’t load at all, you can’t test any of the inputs or outputs on it, or if an application crashes early in its use, you cannot try scenarios after that. Exploratory testing is your chance to quickly check that everything is testable and ready to go. You can raise issues with the development team while preparing comprehensive tests rather than delaying the project when you are ready.
By the end of exploratory testing, the aim is to be able to conduct more thorough testing later. You should come away with detailed knowledge of the interface, all inputs and outputs that were implemented in this version, the configuration options, and the basic functionality. What’s missing are the details and interactions, which we will discuss in Chapter 5, Black-Box Functional Testing.
Exploratory testing is a quick, fun way to get to know a new feature, which doesn’t require the rigor of the full test plans you will design later. The challenge at this early stage is that you know very little about the product, but you can use that to your advantage, as described in the next section.
