Automation – metrics and measurements
Often, when the automation is carried out only after the entire product is ready, the number of times the automation execution is carried out is restricted due to the lack of time. On adding the development and maintenance effort to this, the returns on the investment in automation drastically reduce and sometimes move to negative.
To make an effective investment, it is critical to start automation early on for the prioritized features, based on the release schedule.
A regular tracking and monitoring system will accentuate the measurements and course corrections as required. Some of the key metrics to track are automation coverage, automation index, framework and scripts reusability, automation scripting and execution productivity, and automation ROI.
Automation coverage
This is usually measured as the number of automatable test cases out of the total number of test cases in the Regression and Functional test suite. An average of 70%-80% coverage...