Trading off precision versus brittleness in automated testing

In manual testing, the checks are as precise as the test plan specifies and the experience of the tester performing them. In automated testing, you have to decide the level of detail of the checks you perform, and there is a trade-off between the precision of your tests and how susceptible they are to break in the future, as shown here:

Figure 5.5 – Trading off precision versus resilience in tests

The more precise and detailed you make your tests, the slower and more brittle they will be, liable to break due to small or inconsequential changes. However, in making them faster and more resilient, they will generally become more vague or superficial. While there is no escaping that trade-off, some guidelines can help.

For system testing, you can apply the same rules as you did for the specification – test the behavior, not the implementation. It doesn’t matter about the internal...