Case study
The two pilot teams at Tailwind Gears have achieved a much higher delivery lead time and deployment frequency thanks to the DevOps practices that have been applied. The mean time to restore is also much better because the release pipelines help ship fixes faster. However, the change failure rate has dropped. Releasing more frequently also means that more deployments fail and finding bugs in the code is hard. The quality signals that come from the automated test suites are just not reliable enough and fixing one bug often introduces another bug in another module. There are still many parts of the application that need manual testing – but with one QA engineer in the team, this was not an option. So, some of these parts have been replaced with UI tests, while others have just been dropped.
To evaluate the test portfolio, the teams must introduce a test taxonomy and include reporting in their pipelines. The QA engineers in the team are responsible for the taxonomy...