Software QA FYI - SQAFYI

Is Quality Negotiable

By: Crispin, Lisa

When my husband and I decided to put an addition on our house, we chose to include a basement. We signed a detailed contract with our contractor which specified many little details. We thought we read this carefully. When the basement was built and a hole cut for the door, the contractor pointed out that he had neglected to include the door itself in the contract. We had access to the new basement – it was functional – just no way to close it off if we wanted. Since the door was not included, we would either have to do without it, or pay extra. Naturally, we had assumed there would be a door to the basement room which we could open and shut. But since we had not specified this, the contractor hadn’t included the price of the door or the labor to install it in his price. We couldn’t expect the contractor to just give us a free door. How nice it would have been if someone else had looked at the contract with me and asked, “There isn’t a door specified here, don’t you want one?” Then I could have decided whether or not to spend the money – it wouldn’t have been a surprise later I’ve participated in XP projects where I’ve seen this type of thing happen. (OK, it happens in all software projects, no matter what practices are used). For example, the customer has a story for an add screen, and just assumes the developers know he also wants the ability to update, read and delete. Or maybe there’s a story for a login screen with authentication, but nothing about what should happen if the same user logs in twice. At the end of the iteration, an exception thrown by having the same user log in twice looks like a defect.

As a tester and quality assurance engineer of long experience, I’m something of a tyrant about quality. I have my own standards which naturally I think everyone should follow. When I started working on XP projects, I realized it wasn’t about MY quality standards – it was the customers’.

Here’s an example. Say we have a startup company as our customer. For now, they just need their system up and running to show to potential investors. They just need a system that’s available one or two hours a day for demos. They aren’t looking for a bulletproof 24x7 production server. In fact, they can’t afford to PAY for a bulletproof system right now. They’d rather have more features to show off, even if they might not handle a high level of throughput. It would probably take significantly more time and /or resources to produce a system with guaranteed stability. If the customer isn’t willing to pay the price, they can’t expect to get it for free.

In XP, the customer’s role is to make business decisions, not to be a quality expert. Face it, some people are always on the “happy path”... just as my husband and I were when we signed a contract with our builder for our home addition. As the tester, I feel it’s my responsibility to help the customer make conscious decisions about quality during the planning process. If the customer is clear about his acceptance criteria, and these are reflected accurately in the acceptance tests, we’re much more likely to achieve the level of quality the customer wants, without giving our time away for free.

Internal and External Quality In Extreme Programming Explained, Kent Beck writes:

“There is a strange relationship between internal and external quality. External quality is quality as measured by the customer. Internal quality is quality as measured by the programmers.”

He goes on to explain the human effect on quality:

“If you deliberately downgrade quality, your team might go faster at first, but soon the demoralization of producing crap will overwhelm any gains you temporarily made from not testing, or not reviewing, or not sticking to standards.”

In this light, it looks as if we should always strive for the highest standard of quality. This would of course make me very happy. But is the customer willing to pay for it?

I think the important concept here is the difference between internal and external quality. Whenever I meet someone who works in an XP environment, they always tell me that one of the reasons they love coming to work each morning is they know they’ll be allowed to do their best work. If you take that away, XP won’t work. It’s good to have 100% successful unit tests. In the long run, it speeds up development time. Internal quality should be a given.

External quality can be defined as a set of features, for example:

• Whenever the user makes a mistake, a userfriendly error screen appears • It’s impossible to crash the server via the user interface • The system can handle a hundred concurrent logins

• The system will stay up 99.995% of the time Negotiating with the customer on external quality doesn’t mean skimping on acceptance tests or deliberately producing unstable code. It means that the customer asks for a certain standard of quality and pays for it. If they want a system to handle all exceptions, that should be in the story – or multiple stories. Story one says to implement this functionality; story two says to make the functionality work with N concurrent users hammering it.

The XP Tester as Quality Assurance Engineer The XP books say that the customer writes the test. In Extreme Programming Explained, Kent Beck says customers need to ask themselves, “What would have to be checked before I would be confident this story was done?” This very question implies tests that check for intended functionality, or what my boss calls “Happy

Full article...

Other Resource

... to read more articles, visit

Is Quality Negotiable