The ROI of Test Automation
By: Michael Kelly
Summary: It is widely known that software inspections are a cost-effective approach for finding defects in source code as well as other project documents such as requirements specifications. You can take your inspection process to the next level by using inspections and the resulting data for process improvement throughout your software organization. Lawrence Day presents a basic process flow for inspecting source code and documentation and the keys to implementing a cost-effective inspection approach. Then, he offers a proven approach for using the inspection data to identify process and product improvement opportunities. By viewing inspections a part of your development process, you'll learn to see inspections as a valuable improvement tool. • The basic software inspection process, paths, and benefits • Inspections as a process improvement process • A list of potential inspection product and process improvement opportunities.
Introduction
With the exception of my first project team out of college, in every project team since, I’ve had to explain
either what automated testing is, why we need to do it, or what value it brings to the project. This can be
difficult as each time I explain automation, I have to battle different preconceptions and experiences.
Sometimes I even find myself having to develop a basic understanding of testing and software
development. Wouldn’t it be great if every project manager had to have a testing background or at least
actual development experience? Probably not, I’m sure that would introduce all sorts of other problems.
The point is it is a unique and difficult battle every time.
Educating people about automated testing requires time and mutual respect. Normally both of these are
lacking on projects that are running late, are shot on budget, and have high visibility. Often automation is
thought of as way to reduce the costs of testing, increase coverage, and shorten the testing cycles
required for the project.
This paper looks at the different types of test automation offers guidelines for how to calculate their return
on investment. The topic of test automation is a large one and because every context for which
automation is implemented is different, I cannot offer bulletproof equations that will work for every project.
What I can offer are some general guidelines, some ideas for maximizing your return on investment, and
one or two examples of ROI calculations that are commonly used. The purpose of this article is to better
enable you to communicate the value of automation and determine what types of automation will yield the
greatest return in your specific context.
Why you need ROI analysis
In general, automated testing involves higher upfront costs for a project while providing reduced
execution costs down the road. Performing return on investment (ROI) analysis on each automation
project will determine a simple approximation of cost, will help determine upfront what types of
automation you want for the project, what tools will be required, and what level of skill will be required for
the testing resources for the project. Not only does ROI serve as a justification for effort, but also a
necessary piece of the planning process for the project.
It has been my experience that projects that do not perform ROI calculations upfront do not fully
understand the costs of their automation effort, what types of automation they could be doing vs. what
they are doing, and what strategies to follow to maximize their return. The projects I’ve worked on where
the ROI for the effort was looked at up front had better and more hardware and software resources (due
to justifications). They also had more focus for test planning and test case/script development, and
achieved more goals then those that did not.
The classic example for calculating ROI for automation
Without wasting more time, let’s look at the classic example for how to calculate the ROI for an
automation effort. This classic example (which is a combination of the examples found in the articles by
Varghese and Hoffman referenced below) for ROI is calculated by dividing the net benefits of the
automation effort by the cost of the automation effort, or ROI= Benetfit/Cost
Traditionally the costs used are the costs for automation (hardware, software and licenses, training, time
to produce scripts, etc…) The benefits are calculated by looking out over a time interval at the savings
automated testing offers over manual testing (reduced time to execute tests, ability to test 24 hrs a day,
etc…).
Using that model, we could calculate the ROI for a given project the following way. Let’s look at the
automation of testing for a small website that gets weekly content updates. We would start by looking at
the costs for the effort in terms of time.
Full article...
Other Resource
... to read more articles, visit http://sqa.fyicenter.com/art/
|