background image
<< What is Automated Testing Good For | Usability Testing >>

Non-Functional Testing

<< What is Automated Testing Good For | Usability Testing >>
N o n - f u n c t i o n a l T e s t i n g
Testing the design
Requirements, design and technical specifications can be tested in their own right.
The purpose of evaluating a specification is threefold:
To make sure it is accurate, clear and internally consistent (verification)
Evaluating how well it reflects reality and what the end-user expects (validation)
Making sure it is consistent with all previous and subsequent phases of the project
The technical specification is an embodiment of the requirements which should then flow through
to subsequent phases like production and testing. If the requirements are poorly specified then not
only will the product be inadequate but will also be incredibly difficult to verify.
If the technical specification is out of synch with the requirements then it is likely that the
development team will be well on its way to producing the wrong product. Because these
documents are often produce in parallel (i.e. the steps in the waterfall model overlap) it is very
common for discrepancies to creep in.
Each assertion in a specification should be reviewed against a list of desirable attributes:
Specific it is critical to eliminate as much uncertainty as possible, as early as possible.
Words like "probably", "maybe" or "might" indicate indecision on the part of the
author and hence ambiguity. Requirements including these words should be either
eliminated or re-written to provide some kind of surety as to the desired outcome.
Measurable a requirement which uses comparative words like "better" or "faster"
must specify a quantitative or qualitative improvement must do so with a specific
value (100% faster or 20% more accurate).
Testable in line with the above, a requirement should be specified with some idea of
how it should be evaluated. A requirement which is not testable is ultimately not
`provable' and cannot therefore be confirmed either positively or negatively.
Consistent- if one requirement contradicts another, the contradiction must be resolved.
Often splitting a requirement into component parts helps uncover inconsistent
assumptions in each, which can then be clarified.
Clear - requirements should be simple, clear and concise. Requirements composed of
long-winded sentences or of sentences with multiple clauses imply multiple possible
outcomes and so lack clarity. Split them up into single statements.
Exclusive specs should not only state what will be done, but explicitly what will not
be done. Leaving something unspecified leads to assumptions.
Further, it is important to differentiate requirements from design documents. Requirements should
not talk about "how" to do something and a design specs should not talk about "why" to do