sqaMethods Approach to Designing a Testing Automation System
By: Leopoldo Gonzalez
Building a Testing Automation System
While there may be many ways of building a testing automation system and
most implementations can perform a certain level of automation, I believe that
ultimately the automation system should give testers the capability of an easy transition
from a manual test script to an automated one. It is up to the automation engineer to
hide the complexities of code behind an elegant set of instructions that manual testers
This paper is a collection of thoughts and approaches that describe my
experience in building such a system.
Goals for an Automation System
Early in my testing automation career I realized that in order for an automation
system to be successful, it had to provide five basic functions. Without these
capabilities, the automation system would lack in flexibility and reliability.
1. The automated script must be able to be executed as a single entity.
This simply means that the tester must be able to pick a script from a list of
scripts and choose to run it alone. The script must know that it is not
running as part of a suite and if needed, be able to gather its own
2. The automated script must be able to be executed as a suite.
The same script must know that when running as part of a suite, it cannot
stop and gather information from the user as it would defeat the purposed of
running unattended. Also there are times when you donít want the script to
exit the application under test only to be re-launched by the next script in thesuite. This is a waste of time and only prolongs the execution time. The
script must know not to exit but to return to a HOME state.
3. The script must be data driven.
The data the script uses must reside somewhere other than in the script
itself. There should never be hard coded data inside the script. Data driven
scripts allow the tester to create multiple test case scenarios without having
to modify the code itself.
4. All of the test scripts in a suite must execute.
This means that the failure of one script should not affect the execution of
the next one. If during the execution of a script, the application aborts, the
script must be able to recover from that failure and attempt to continue with
the next script in the suite.
5. The automated script should use the same logic flow as the manual script
The transition between a manual script and an automated one will be a lot
easier if when converting a script, the tester can follow the same logic flow
and use some of the same language he used before.
While each of the above goals is worthy of its own analysis, this paper will not
cover all five goals, however I will talk a little on the last one.
Begin with the End in mind
Letís begin with a visualization of what a testing automation system should be
for a QA shop. This system should be treated as a useful tool to be used during the QA
test cycle. It should be the first line of tests to be executed when a new build is
created (smoke test). It should be used as a regression test mechanism (regression
test), it should be updated with new test cases for new functionality and finally, if
performance test is included, it should be used to measure the systemís performance
under varying loads.
But the question is, how do get from where you are to a system that is part of a
well oiled QA testing machine?
The Three Tier System
The three tier system of software development is very well known and utilized
for big projects. It is modular; it separates functions at their right domain area and is
just about the right size to avoid too much complexity.
By the same token, in my experience I found out that this model works fine for
testing automation systems as well. The only difference is that rather that breaking the
model down to SQL, Business Objects and GUI as in the traditional three tier approach,
a testing automation system should be made out of Common Utilities, Business Objects
and Test Cases.
The methods in the Common Utilities would be inherited by the Business
Objects, and these would be inherited by the Test Case scripts.
This model allows us to grow capability at the right level of abstraction. If I
need a method that would be useful to all of the automation system, I would write it at
the Common Utilities tier. However, if the method falls inside one of the business
domains, such as ordering, navigation, admin, etc., then I would write it at the Business
Object tier. The last tier to build is the actual test script. The syntax used by the test
script should follow closely the logic used in the manual test script. Be aware that you
will not be able to completely remove the flavor of the language from your testing
automation tool, but at least you will have a system on which a tester can venture out
and try to develop an automated script by himself even though he may not be a
Applying the Concept
To see the benefit of this model, letís examine a manual test case and create the
necessary components that will allow us to mirror the syntax of the manual test case.
... to read more articles, visit http://sqa.fyicenter.com/art/