improve regression testing effectiveness by 30%?
The goal is to ensure application quality by detecting most regressions while running fewer tests. It seems impossible to win simultaneously on quality & time? Letís see how impact analysis can increase testing effectiveness by 30%.
It is often a headache. Regression testing represents an important part of the overall test effort on each application release. They are required because they detect problems that have been undetected upstream. However they also often miss bugs which are found after deployment when the cost to fix them is usually estimated to be at least 10 times higher (diagnostic, fix, test, deploy, business impact, etc.).
Regression testing effectiveness is required to:
- Reduce the risk of bugs in each release and thus select the right tests to execute in the time frame dedicated to regression testing.
- Be responsive to business needs with faster release and smaller release. This requires also faster regression testing and to be able to design effective regression test campaigns according to the content of each version.
- Adapt to financial constraints
How much regression testing?
For most projects, even small ones, the number of tests required to ensure that there are no regressions can quickly become unmanageable. The following diagram illustrates this situation if all tests created for a specific are always added to the regression test suite.
to define a regression test strategy?
It is recommended to include a specific approach for regression testing in the test strategy in order to limit the number of tests, usually it includes mainly two types of tests:
1. Tests related to major features of the product with a priority regarding requirements and risks.
2. Tests related to features that have been proven to be less robust (where bugs and/ or regressions were found in previous releases).
However, if these criteria reduce the number of tests, they do not adapt the strategy to each software release. Thus it leads to execute always the same test suites and the result is often that regressions are not detected. How to improve?
Automated tests appear to be the Holy Grail: execute all tests, on every release within a limited timeframe and for a fraction of the costs. A greater number of tests should lead to a better functional coverage and thus a better quality.
Reality is quite different because statistics made on our platform show that 80% of projects do not have any automated functional tests, 5% have intensive automation strategy, and 15% focused and limited test automation.
Why so few automated tests? Customers explain that itís not always faster and cheaper because test maintenance is a significant burden. Test automation could be the right choice but it has to be done with a well structured approach.
Impact analysis to optimize software testing
Why impact analysis?
When a new release of an application is done, whether corrections, changes, or new features, development teams are making changes to the application code. It is these changes that generate regressions on existing features.
... to read more articles, visit http://sqa.fyicenter.com/art/