<< Performance Testing | Software in Many Dimensions >>

Test Planning

<< Performance Testing | Software in Many Dimensions >>

T e s t P l a n n i n g

The Purpose of Test Planning

As part of a project, testing must be planned to ensure it delivers on its expected outcomes. It
must be completed within a reasonable time and budget.

But test planning represents a special challenge.

The aim of test planning is to decide where the bugs in a product or system will be and then to
design tests to locate them. The paradox is of course, that if we knew where the bugs were then
we could fix them without having to test for them.

Testing is the art of uncovering the unknown and therefore can be difficult to plan.

The usual, naïve retort is that you should simply test "all" of the product. Even the simplest
program however will defy all efforts to achieve 100% coverage (see the appendix).

Even the term coverage itself is misleading since this represents a plethora of possibilities. Do you
mean code coverage, branch coverage,or input/output coverage ? Each one is different and each
one has different implications for the development and testing effort. The ultimate truth is that
complete coverage, of any sort, is simply not possible (nor desirable).

So how do you plan your testing?

At the start of testing there will be a (relatively) large number of issues and these can be
uncovered with little effort. As testing progress more and more effort is required to uncover
subsequent issues.

The law of diminishing returns applies and at some point
the investment to uncover that last 1% of issues is
outweighed by the high cost of finding them. The cost of
letting the customer or client find them will actually be
less than the cost of finding them in testing.

The purpose of test planning therefore is to put together
a plan which will deliver the right tests, in the right order,
to discover as many of the issues with the software as
time and budget allow.

Risk Based Testing

Risk is based on two factors the likelihood of the problem occurring and the impact of the
problem when it does occur. For example, if a particular piece of code is complex then it will
introduce far more errors than a simple module of code. Or a particular module of code could be
critical to the success of the project overall. Without it functioning perfectly, the product simply
will not deliver its intended result.

Both of these areas should receive more attention and more testing than less 'risky' areas.

But how to identify those areas of risk?

Figure 6: Typical defect discovery rate

Time

Defects