Build Automated QA into Your Code

Interview Questions

Why are there Bugs in Software?

Build Automated QA into Your Code

Obviously, to build defect-free code, you have to be able to test your code. In addition to including a testing plan/strategy into the implementation, you should design specific code to provide for full, automated testability.

The most effective testing we use is fully automated or regression testing. This is a series of fully automated tests that are run after each build of a program. The tests are designed to exercise every part of the program, and produce a success/failure report. The idea is to use the power of the computer to make sure that the program hasn't been adversely affected by a change.

If the design is well structured, most changes should not have side effects. The purpose of these automated tests is to provide insurance that the coding assumptions are valid, and that everything else still works. By making the tests completely automated, they can be run frequently and provide prompt feedback to the engineer.

If tests are run by manually testing the program, we have the chance of human error missing a problem. Manual testing is also very expensive, usually too expensive to run after every change to a program.

There are a number of commercial testing tools available which are designed to help you automate your testing, particularly in GUI environments such as Windows. Although they are no doubt better than manual testing, we have not found them to be effective, for a number of reasons.

By building support for automated testing into your program, you can approach 100% automated testing. Without this customized, built-in testability, you will be lucky to achieve 35% automated testing, even with the best commercial QA testing tool. We recommend that you budget five percent of total engineering time to creating support for automated QA testing.

Of course, each new piece of code should have a corresponding set of tests, added at the same time as the code is added, for the automated QA suite.

In order for fully automated execution of testing to be of value, the tests that are automatically executed and checked must cover the software fully. To the extent that they don't, running the tests doesn't tell you anything about the part of your software that wasn't exercised by the testing. (This is true for all testing, whether automated or manual.)

(Continued on next question...)