Test Automation
Test Automation
Organisations often seek to reduce the cost of testing. Most organisations aren't comfortable with
reducing the amount of testing so instead they look at improving the efficiency of testing. Luckily,
there are a number of software vendors who claim to be able to do just this! They offer
automated tools which take a test case, automate it and run it against a software target repeatedly.
Music to management ears!
However, there are some myths about automated test tools that need to be dispelled :
Automated testing does not find more bugs than manual testing an experienced manual tester
who is familiar with the system will find more new defects than a suite of automated tests.
Automation does not fix the development process as harsh as it sounds, testers don't create
defects, developers do. Automated testing does not improve the development process
although it might highlight some of the issues.
Automated testing is not necessarily faster the upfront effort of automating a test is much
higher than conducting a manual test, so it will take longer and cost more to test the first
time around. Automation only pays off over time. It will also cost more to maintain.
Everything does not need to be automated some things don't lend themselves to automation,
some systems change too fast for automation, some tests benefit from partial automation
you need to be selective about what you automate to reap the benefits.
But, in their place, automated test tools can be extremely successful.
The funniest business case I have ever seen for automated test tools ran like this :
Using manual testing, we find X number of defects in our software
It costs $Y to find and fix these defects each year (developer + tester time)
We can buy an automated test tool for $Z/year
Since $Z < $Y we should buy the tool and, as a bonus, it will find < X defects
So, not only are you comparing the one off cost for buying tools (without set-up or maintenance)
with the cost of manual testing, but the tool will do away with the manual testers as well the ones
who find all those pesky defects!
The Hidden Cost
The hidden costs of test automation are in its maintenance.
An automated test asset which can be written once and run many times pays for itself much
quicker than one which has to be continually rewritten to keep pace with the software.
And there's the rub.
Automation tools, like any other piece of software, talk to the software-under-test through an
interface. If the interface is changing all the time then, no matter what the vendors say, your tests
will have to change as well. On the other hand, if the interfaces remain constant but the underlying
functional code changes then your tests will still run and (hopefully) still find bugs.
Many software projects don't have stable interfaces. The user-facing interface (or GUI) is in fact the
area which has most change because its the bit the users see most. Trying to automate the testing
for a piece of software which has a rapidly changing interface is a bit like trying to pin a jellyfish to
the wall.
15
