background image
<< Test Case Identification | Test Planning Summary >>

Test Estimation

<< Test Case Identification | Test Planning Summary >>
Test Estimation
Once you have prioritised the test cases you can estimate how long each case will take to execute.
Take each test case and make a rough estimate of how long you think it will take to set up the
appropriate input conditions, execute the software and check the output. No two test cases are
alike but you can approximate this step by assigning and average execution time to each case and
multiplying by the number of cases.
Total up the hours involved and you have an estimate of testing for that piece of software.
You can then negotiate with the Project Manager or your product manager for the appropriate
budget to execute the testing. The final cost you arrive at will depend on the answers to a number
of questions, including : how deep are your organisation's pockets? how mission critical is the
system? how important is quality to the company? how reliable is the development process?
The number will almost certainly be lower than you hoped for.
In general ­ more iterations are better than more tests. Why? Developers don't often fix a bug at
the first attempt. Bugs tend to cluster and finding one may uncover more. If you have a lot of bugs to
fix you will have to have multiple iterations to retest the fixes. And finally, and honestly, in a mature
software development effort, many tests will uncover no bugs!
Refining the plan
As you progress through each cycle of testing you can refine your test plan.
Typically, during the early stages of testing not many
defects are found. Then, as testing hits it stride, defects
start coming faster and faster until the development team
gets on top of the problem and the curve begins to
flatten out. As development moves ahead and as testing
moves to retesting fixed defects, the number of new
defects will decrease.
This is the point where your risk/reward ratio begins to
bottom out and it may be that you have reached the
limits of effectiveness with this particular form of testing.
If you have more testing planned or more time available,
now is the time to switch the focus of testing to a
different point in your outline strategy.
Cem Kaner said it best, "The best test cases are the ones that find bugs." A test case which finds
no issues is not worthless but it is obviously worth less than a test case which does find issues.
If your testing is not finding any bugs then perhaps you should be looking somewhere else.
Conversely, if your testing is finding a lot of issues you should pay more attention to it ­ but not to
the exclusion of everything else, there's no point in fixing just one area of the software!
Your cycle of refinement should be geared towards discarding pointless or inefficient tests and
diverting attention to more fertile areas for evaluation.
Also, referring each time to your original outline will help you avoid losing sight of the wood for
the trees. While finding issues is important you can never be sure where you'll find them so you
can't assume the issues that you are finding are the only ones that exist. You must keep a
continuous level of broad coverage testing active to give you an overview of the software while you
focus the deep coverage testing on the trouble spots.
23
Figure 9: Defect discovery rate again
Time
Defects