What is a Serious Bug? Defining a "Material Breach" of a Software License Agreement

By: Cem Kaner

Here's the proposed definition of a material breach of the software contract in July, 1996 (and in several previous drafts).

2B-109 (July) (b) A breach is material if the contract so provides or, in the absence of express contractual terms, if the circumstances, intent of the parties, language of the contract, and character of the breach indicate that the breach caused or may cause substantial harm to the interests of the aggrieved party, or if it meets the conditions of subsection (c) or (d).

(c)A breach is material if it involves:
(1)knowing or grossly negligent disclosure or use of confidential information of the aggrieved party not justified by the license;
(2)knowing infringement of the aggrieved party's intellectual property rights not authorized by the terms of the license and occurring over more than a brief period; or
(3)an uncured, substantial failure to pay a license fee when due which is not justified by an existing, colorable dispute about whether payment is due.
(d)A material breach occurs if the aggregate effect of the nonmaterial breaches by the same party satisfy the standards for materiality.

Look closely at (c), which defines a material breach by listing examples. All of these protect the publisher from breaches of the contract by the customer. For example, the customer is in trouble if s/he (1) discloses the publisher's confidential information, (2) pirates the software, or (3) doesn't pay for the program. None of these helps the customer argue that a given bug is material. The September and November Drafts

The September, 1996 draft added a further clause to the list. A breach is also material if it involves:

a failure to perform in a manner consistent with express performance standards;

In September, I asked Ray Nimmer what this means. What is an "express performance standard?" Is it any precise, verifiable statement about the program that the seller makes to the customer? For example, can the customer get a refund if the program fails to match the written documentation?

He said, no, that's not what he meant. He was referring to the specific promises that were contained in a negotiated contract, that said that the program must do specified things or have specified characteristics.

So I said that this is unfair. What about customers who don't have negotiated contracts that specify all the right things? What about mass-market customers? If the program doesn't work, don't these customers have any recourse?

Ray said that this is a genuine problem. But he doesn't know how to solve it. How should we define material defect when the bug is not a direct nonconformity with an important part of the specification?

Then he said to me that I was the person who kept raising these issues, so maybe I could take a crack at drafting the definition.

And I promised that I would. Not that I claim any particular experience or skill in statutory drafting. I hope and trust that you will read this proposal for the idea it contains and recognize that its awkwardnesses will be cleaned up during the Article 2B review process.

How Should We Define a Serious Defect?
Failure to conform to specifications is a common theme in legal books, but many of the software development contracts provide vague, incomplete specifications that will change over time without being updated in the contract itself. Discussions within the software development community consistently recognize that most failures in commercial software products are due to errors in the specifications or requirements. A widely used number is that 80% of the money spent fixing or dealing with software problems can be traced back to requirements errors.

As a result, texts that focus on software errors don't limit themselves to failure to meet a specification (this type of failure is called nonconformance). Here are some examples from well respected texts in the field:

IEEE ( 1989), IEEE Standard Dictionary of Measures to Produce Reliable Software, ANSI/IEEE Standard 982.1-1988, p. 13:
Defect: A product anomaly. Examples include such things as (1) omissions and imperfections found during early life cycle phases and (2) faults contained in software sufficiently mature for test or operation. See also fault.
IEEE (1994), IEEE Standard Classification for Software Anomalies, IEEE Standard 1044-1993, p. 3.
Anomaly: Any condition that deviates from expectations based on requirements specifications, design documents, user documents, standards, etc., or from someone's perceptions or experiences. Anomalies may be found during, but not limited to, the review, test, analysis, compilation, or use of software products or applicable documentation.

Grady, Robert B. & Caswell, Deborah, L. (1987) Software Metrics: Establishing a Company-Wide Program. PTR Prentice-Hall, p. 78

A defect is any flaw in the specification, design, or implementation of a product. . . . If a flaw could not possibly have been detected, or if it could have been detected and would not have been corrected then it is an enhancement. Defects do not include typographical or grammatical errors in engineering documentation.

Ishikawa, Kaoru (translated by David J. Lu) (1985) What is Total Quality Control? The Japanese Way, Prentice-Hall. (Ishikawa is the leading Japanese quality control theoriest):

On page 46 ff. he explains why he doesn't trust quality as measured in terms of compliance with standards and specifications. The problem is that these are not true measures of the quality of the product. He works through an excellent example dealing with a role of newsprint. The true measure of quality is whether the paper rips while on the rotary press. Published standards, in terms of such things as tensile strength, provide only secondary measures of how the product will perform in the field. Continuing, on page 56, "There are no standards-whether they be national, international, or company-wide-that are perfect. Usually standards contain some inherent defects. Consumer requirements also change continuously, demanding higher quality year after year. Standards that were adequate when they were first established, quickly become obsolete. [¶]We engage in QC to satisfy customer requirements"

Jones, Capers (1991), Applied Software Measurement, McGraw-Hill, page 273.
A software defect is simply a bug which if not removed would cause a program or system to fail or to produce incorrect results. Note: the very common idea that a defect is a failure to adhere to some user requirements is unsatisfactory because it offers no way to measure requirements defects themselves, which constitute one of the larger categories of software error.

Full article...

Other Resource

... to read more articles, visit http://sqa.fyicenter.com/art/