Software QA FYI - SQAFYI

The ROI of Test Automation

By: Michael Kelly

Summary: It is widely known that software inspections are a cost-effective approach for finding defects in source code as well as other project documents such as requirements specifications. You can take your inspection process to the next level by using inspections and the resulting data for process improvement throughout your software organization. Lawrence Day presents a basic process flow for inspecting source code and documentation and the keys to implementing a cost-effective inspection approach. Then, he offers a proven approach for using the inspection data to identify process and product improvement opportunities. By viewing inspections a part of your development process, you'll learn to see inspections as a valuable improvement tool. • The basic software inspection process, paths, and benefits • Inspections as a process improvement process • A list of potential inspection product and process improvement opportunities.

Introduction With the exception of my first project team out of college, in every project team since, I’ve had to explain either what automated testing is, why we need to do it, or what value it brings to the project. This can be difficult as each time I explain automation, I have to battle different preconceptions and experiences. Sometimes I even find myself having to develop a basic understanding of testing and software development. Wouldn’t it be great if every project manager had to have a testing background or at least actual development experience? Probably not, I’m sure that would introduce all sorts of other problems. The point is it is a unique and difficult battle every time. Educating people about automated testing requires time and mutual respect. Normally both of these are lacking on projects that are running late, are shot on budget, and have high visibility. Often automation is thought of as way to reduce the costs of testing, increase coverage, and shorten the testing cycles required for the project. This paper looks at the different types of test automation offers guidelines for how to calculate their return on investment. The topic of test automation is a large one and because every context for which automation is implemented is different, I cannot offer bulletproof equations that will work for every project. What I can offer are some general guidelines, some ideas for maximizing your return on investment, and one or two examples of ROI calculations that are commonly used. The purpose of this article is to better enable you to communicate the value of automation and determine what types of automation will yield the greatest return in your specific context. Why you need ROI analysis In general, automated testing involves higher upfront costs for a project while providing reduced execution costs down the road. Performing return on investment (ROI) analysis on each automation project will determine a simple approximation of cost, will help determine upfront what types of automation you want for the project, what tools will be required, and what level of skill will be required for the testing resources for the project. Not only does ROI serve as a justification for effort, but also a necessary piece of the planning process for the project. It has been my experience that projects that do not perform ROI calculations upfront do not fully understand the costs of their automation effort, what types of automation they could be doing vs. what they are doing, and what strategies to follow to maximize their return. The projects I’ve worked on where the ROI for the effort was looked at up front had better and more hardware and software resources (due to justifications). They also had more focus for test planning and test case/script development, and achieved more goals then those that did not. The classic example for calculating ROI for automation Without wasting more time, let’s look at the classic example for how to calculate the ROI for an automation effort. This classic example (which is a combination of the examples found in the articles by Varghese and Hoffman referenced below) for ROI is calculated by dividing the net benefits of the automation effort by the cost of the automation effort, or ROI= Benetfit/Cost

Traditionally the costs used are the costs for automation (hardware, software and licenses, training, time to produce scripts, etc…) The benefits are calculated by looking out over a time interval at the savings automated testing offers over manual testing (reduced time to execute tests, ability to test 24 hrs a day, etc…).

Using that model, we could calculate the ROI for a given project the following way. Let’s look at the automation of testing for a small website that gets weekly content updates. We would start by looking at the costs for the effort in terms of time.

Full article...

Other Resource

... to read more articles, visit

The ROI of Test Automation