Non-Regression Test Automation (Part 2)
By: Doug Hoffman
Automated software testing has historically meant having the computer run individually crafted test cases that perform the same exercises as manual tests, only run by a machine. The principle advantages for these automated tests are repeatability and speed. The principle disadvantages are that they are relatively more expensive to create than manual tests, require more maintenance than manual tests, and are more limited in the specificity of things they can compare compared with manual tests.
Part 1 of this presentation describes another way to approach test automation: to test things that cannot be tested manually. Extending the scope of testing in this way allows checking for errors that might not be found otherwise or even conceived of. These tests enable us to focus on learning about the software, can go behind the UI to extend our reach, are not limited to doing the same thing each time (although even random sequences can be repeated), and can perform huge numbers of iterations and combinations unthinkable using manual testing or automated regression tests. These tests are usually high-volume, quick-hit, or abstracted one or two levels from the user interface, which substantially reduces maintenance costs. This approach also encourages checking broader classes of test outcomes, thus improving the types of errors that can be discovered.
Part 2 describes oracle mechanisms that enable testers to take advantage of non-regression automation. The oracles determine whether the software’s behavior appears to be normal or erroneous. The oracles allow non-regression tests to vary their behavior and still have predictable, checkable outcomes. This session presents over a dozen different types of oracle mechanisms.
* How to use automated tests to extend testers’ capabilities
* The virtues and drawbacks to automated regression tests
* Design of automated tests for exploration
* A model for understanding outside influences and hidden outcomes during testing
* Test oracle mechanisms for automated tests
... to read more articles, visit http://sqa.fyicenter.com/art/