background image
<< Ch. 5 Automatically Generating Scripts | Setting Up and Running Pilots >>
<< Ch. 5 Automatically Generating Scripts | Setting Up and Running Pilots >>

Objectives for creating test scripts

Automatically Generating Scripts
5-2
Your objectives for creating test scripts depend on what phase of development the
AUT is in. Early in the development process, and during phases of rapid change in
the AUT, it is more important to flush out severe defects than to create scripts that
provide high coverage and have longevity.
Traditionally, engineers did not bother to automate testing at the beginning stages of
product development because of the high cost of maintaining scripts. With
TestFactory, the cost of script maintenance is so low that you can incorporate
automation in your development cycle as soon as you have a user interface. Early in
product development, you can run Pilots as a smoke test to flush out defects quickly.
The Pilot results include defect scripts that uncover severe bugs such as AUT
crashes, Visual Basic run-time errors, and assertions. You can use Pilot runs early in
the development cycle and against each new build of the AUT to spot areas of code
instability. You can run Pilots after fixing severe defects to uncover new defects and
to ensure that the AUT has not regressed.
The Pilot run results can also include UAW (unexpected active window) scripts. If
an unexpected active window opens during a Pilot run, TestFactory pulls the
running script segment and saves it as a UAW script. You can examine a UAW script
to determine what steps a Pilot performed to activate an unexpected window, and
use the information to improve the application map.
Although you continue to look for defects as the AUT matures and stabilizes, at
some point you become more concerned with how completely your scripts test the
product and how well the AUT meets functional requirements. As functional areas
of the AUT stabilize, you can begin running Pilot scenarios to simulate action
sequences that users are likely to perform. You can also compose Pilot mix-ins to test
the interaction of several different functions or to occasionally introduce a new
functional element to an otherwise ordered Pilot scenario.
Late in the development cycle, once the AUT is free of severe defects and has been
proven to meet its functional requirements, you can set up and run Pilots that check
for memory-related defects in the AUT. Pilots can generate defect scripts that
uncover memory and resource leaks, invalid memory access errors, memory
overwrites, uninitialized memory reads, and memory access beyond the bounds of
an array.