<< Software in Many Dimensions | Test Estimation >>

Test Case Identification

<< Software in Many Dimensions | Test Estimation >>

The next step is to identify test cases which 'exercise' each of the elements in your outline.

This isn't a one-to-one relationship. Many tests might be required to validate a single element of
your outline and a single test may validate more than one point on one axis. For example, a single
test could simultaneously validate functionality, code structure, a UI element and error handling.

In the diagram shown here, two tests have been highlighted in
red : "A" and "B".

Each represents a single test which has verified the behaviour
of the software on the two particular axes of "input data
space" and "hardware environment".

For each point in each axis of your outline decide upon a test
which will exercise that functionality. Note if that test will
validate a different point on a different axis and continue until
you are satisfied you have all points on all axes covered.

For now, don't worry too much about the detail of how you might test each point, simply decide
'what' you will test. At the end of this exercise you should have a list of test cases which represents
your near-to-perfect, almost 100% coverage of the product.

Test Case Selection

Given that we acknowledge we can't achieve a 100% coverage, we must now take a critical eye to
our list of test cases. We must decide which ones are more important, which ones will exercise
the areas of risk and which ones will discover the most bugs.

But how?

Have another look at the diagram above we have two of our axes represented here, "input data
space" and "hardware environment". We have two test "A" and "B" and we have dark splotches,
which denote bugs in the software.

Bugs tend to cluster around one or more areas within one or more axes. These define the areas of
risk in the product. Perhaps this section of the code was completed in a hurry or perhaps this
section of the 'input data space' was particularly difficult to deal with.

Whatever the reason, these areas are inherently more risky, more likely to fail than others.

You can also note that test A has not uncovered a bug it has simply verified the behaviour of the
program at that point. Test B on the other hand has identified a single bug in a cluster of similar
bugs. Focussing testing efforts around area B will be more productive since there are more more
bugs to be uncovered here. Focussing around A will probably not produce any significant results.

When selecting test cases you must try to achieve a balance.

Your aim should be to provide a broad coverage for the majority of your outline 'axes' and deep
coverage for the most risky areas discovered. Broad coverage implies that an element in the
outline is evaluated in a elementary fashion while deep coverage implies a number of repetitive,
overlapping test cases which exercise every variation in the element under test.

The aim of broad coverage is to identify risky areas and focus the deeper coverage of those areas
to eliminate the bulk of issues. It is a tricky balancing act between trying to cover everything and
focusing your efforts on the areas that require most attention.

Figure 8 : Test case identification