Software QA FYI - SQAFYI

sqaMethods Approach to Designing a Testing Automation System

By: Leopoldo Gonzalez

Building a Testing Automation System

While there may be many ways of building a testing automation system and most implementations can perform a certain level of automation, I believe that ultimately the automation system should give testers the capability of an easy transition from a manual test script to an automated one. It is up to the automation engineer to hide the complexities of code behind an elegant set of instructions that manual testers can understand.

This paper is a collection of thoughts and approaches that describe my experience in building such a system.

Goals for an Automation System
Early in my testing automation career I realized that in order for an automation system to be successful, it had to provide five basic functions. Without these capabilities, the automation system would lack in flexibility and reliability. 1. The automated script must be able to be executed as a single entity.
This simply means that the tester must be able to pick a script from a list of scripts and choose to run it alone. The script must know that it is not running as part of a suite and if needed, be able to gather its own information.

2. The automated script must be able to be executed as a suite.
The same script must know that when running as part of a suite, it cannot stop and gather information from the user as it would defeat the purposed of running unattended. Also there are times when you donít want the script to exit the application under test only to be re-launched by the next script in thesuite. This is a waste of time and only prolongs the execution time. The script must know not to exit but to return to a HOME state.

3. The script must be data driven.
The data the script uses must reside somewhere other than in the script itself. There should never be hard coded data inside the script. Data driven scripts allow the tester to create multiple test case scenarios without having to modify the code itself.

4. All of the test scripts in a suite must execute.
This means that the failure of one script should not affect the execution of the next one. If during the execution of a script, the application aborts, the script must be able to recover from that failure and attempt to continue with the next script in the suite.

5. The automated script should use the same logic flow as the manual script
The transition between a manual script and an automated one will be a lot easier if when converting a script, the tester can follow the same logic flow and use some of the same language he used before.
While each of the above goals is worthy of its own analysis, this paper will not cover all five goals, however I will talk a little on the last one.

Begin with the End in mind
Letís begin with a visualization of what a testing automation system should be for a QA shop. This system should be treated as a useful tool to be used during the QA test cycle. It should be the first line of tests to be executed when a new build is created (smoke test). It should be used as a regression test mechanism (regression test), it should be updated with new test cases for new functionality and finally, if performance test is included, it should be used to measure the systemís performance under varying loads.

But the question is, how do get from where you are to a system that is part of a well oiled QA testing machine?

The Three Tier System
The three tier system of software development is very well known and utilized for big projects. It is modular; it separates functions at their right domain area and is just about the right size to avoid too much complexity.

By the same token, in my experience I found out that this model works fine for testing automation systems as well. The only difference is that rather that breaking the model down to SQL, Business Objects and GUI as in the traditional three tier approach, a testing automation system should be made out of Common Utilities, Business Objects and Test Cases.

The methods in the Common Utilities would be inherited by the Business Objects, and these would be inherited by the Test Case scripts.

This model allows us to grow capability at the right level of abstraction. If I need a method that would be useful to all of the automation system, I would write it at the Common Utilities tier. However, if the method falls inside one of the business domains, such as ordering, navigation, admin, etc., then I would write it at the Business Object tier. The last tier to build is the actual test script. The syntax used by the test script should follow closely the logic used in the manual test script. Be aware that you will not be able to completely remove the flavor of the language from your testing automation tool, but at least you will have a system on which a tester can venture out and try to develop an automated script by himself even though he may not be a programmer.

Applying the Concept To see the benefit of this model, letís examine a manual test case and create the necessary components that will allow us to mirror the syntax of the manual test case.

Full article...

Other Resource

... to read more articles, visit

sqaMethods Approach to Designing a Testing Automation System