Quality-Related Costs. The Economics of Software Quality

By: Cem Kaner

What is Quality Cost Analysis?
Quality costs are the costs associated with preventing, finding, and correcting defective work. These costs are huge, running at 20% - 40% of sales.3 Many of these costs can be significantly reduced or completely avoided. One of the key functions of a Quality Engineer is the reduction of the total cost of quality associated with a product.

Here are six useful definitions, as applied to software products. Figure 1 gives examples of the types of cost. Most of Figure 1's examples are (hopefully) self-explanatory, but I'll provide some additional notes on a few of the costs:4

* Prevention Costs: Costs of activities that are specifically designed to prevent poor quality. Examples of "poor quality" include coding errors, design errors, mistakes in the user manuals, as well as badly documented or unmaintainably complex code.

* Note that most of the prevention costs don't fit within the Testing Group's budget. This money is spent by the programming, design, and marketing staffs.

* Appraisal Costs: Costs of activities designed to find quality problems, such as code inspections and any type of testing.

* Design reviews are part prevention and part appraisal. To the degree that you're looking for errors in the proposed design itself when you do the review, you're doing an appraisal. To the degree that you are looking for ways to strengthen the design, you are doing prevention.

* Failure Costs: Costs that result from poor quality, such as the cost of fixing bugs and the cost of dealing with customer complaints. * Internal Failure Costs: Failure costs that arise before your company supplies its product to the customer. Along with costs of finding and fixing bugs are many internal failure costs borne by groups outside of Product Development. If a bug blocks someone in your company from doing her job, the costs of the wasted time, the missed milestones, and the overtime to get back onto schedule are all internal failure costs.

* For example, if your company sells thousands of copies of the same program, you will probably print several thousand copies of a multi-color box that contains and describes the program. You (your company) will often be able to get a much better deal by booking press time in advance. However, if you don't get the artwork to the printer on time, you might have to pay for some or all of that wasted press time anyway, and then you may have to pay additional printing fees and rush charges to get the printing done on the new schedule. This can be an added expense of many thousands of dollars.

Some programming groups treat user interface errors as low priority, leaving them until the end to fix. This can be a mistake. Marketing staff need pictures of the product's screen long before the program is finished, in order to get the artwork for the box into the printer on time. User interface bugs - the ones that will be fixed later - can make it hard for these staff members to take (or mock up) accurate screen shots. Delays caused by these minor design flaws, or by bugs that block a packaging staff member from creating or printing special reports, can cause the company to miss its printer deadline.

Including costs like lost opportunity and cost of delays in numerical estimates of the total cost of quality can be controversial. Campanella (1990)5 doesn't include these in a detailed listing of examples. Gryna (1988)6 recommends against including costs like these in the published totals because fallout from the controversy over them can kill the entire quality cost accounting effort. I include them here because I sometimes find them very useful, even if it might not make sense to include them in a balance sheet.

* External Failure Costs: Failure costs that arise after your company supplies the product to the customer, such as customer service costs, or the cost of patching a released product and distributing the patch.

* External failure costs are huge. It is much cheaper to fix problems before shipping the defective product to customers.

Some of these costs must be treated with care. For example, the cost of public relations efforts to soften the publicity effects of bugs is probably not a huge percentage of your company's PR budget. You can't charge the entire PR budget as a quality-related cost. But any money that the PR group has to spend to specifically cope with potentially bad publicity due to bugs is a failure cost.

I've omitted from Figure 1 several additional costs that I don't know how to estimate, and that I suspect are too often too controversial to use. Of these, my two strongest themes are cost of high turnover (people quit over quality-related frustration - this definitely includes sales staff, not just development and support) and cost of lost pride (many people will work less hard, with less care, if they believe that the final product will be low quality no matter what they do.)

* Total Cost of Quality: The sum of costs: Prevention + Appraisal + Internal Failure + External Failure.

What Makes this Approach Powerful?
* Ask the writers: Is this design odd enough that it is causing extra effort to document? Would a simpler design reduce writing time and the number of pages in the manual?

* Ask the training staff: Are they going to have to spend extra time in class, and to write more supplementary materials because of this design?

* Ask Technical Support and Customer Service: Will this design increase support costs? Will it take longer to train support staff? Will there be more calls for explanations or help? More complaints? Have customers asked for refunds in previous versions of the product because of features designed like this one?

* Check for related problems: Is this design having other effects on the reliability of the program? Has it caused other bugs? (Look in the database.) Made the code harder to change? (Ask the programmers.)

* Ask the sales staff: If you think that this feature is very visible, and visibly wrong, ask whether it will interfere with sales demonstrations, or add to customer resistance.

* What about magazine reviews? Is this problem likely to be visible enough to be complained about by reviewers? If you think so, check your impression with someone in Marketing or PR.

Full article...

Other Resource

... to read more articles, visit http://sqa.fyicenter.com/art/