Is Quality Negotiable
By: Crispin, Lisa
When my husband and I decided to put an
addition on our house, we chose to include a
basement. We signed a detailed contract with
our contractor which specified many little
details. We thought we read this carefully.
When the basement was built and a hole cut for
the door, the contractor pointed out that he had
neglected to include the door itself in the
contract. We had access to the new basement –
it was functional – just no way to close it off if
we wanted. Since the door was not included, we
would either have to do without it, or pay extra.
Naturally, we had assumed there would be a door
to the basement room which we could open and
shut. But since we had not specified this, the
contractor hadn’t included the price of the door
or the labor to install it in his price. We couldn’t
expect the contractor to just give us a free door.
How nice it would have been if someone else
had looked at the contract with me and asked,
“There isn’t a door specified here, don’t you
want one?” Then I could have decided whether
or not to spend the money – it wouldn’t have
been a surprise later
I’ve participated in XP projects where I’ve seen
this type of thing happen. (OK, it happens in all
software projects, no matter what practices are
used). For example, the customer has a story for
an add screen, and just assumes the developers
know he also wants the ability to update, read
and delete. Or maybe there’s a story for a login
screen with authentication, but nothing about
what should happen if the same user logs in
twice. At the end of the iteration, an exception
thrown by having the same user log in twice
looks like a defect.
As a tester and quality assurance engineer of
long experience, I’m something of a tyrant about
quality. I have my own standards which
naturally I think everyone should follow. When
I started working on XP projects, I realized it
wasn’t about MY quality standards – it was the
Here’s an example. Say we have a startup
company as our customer. For now, they just
need their system up and running to show to
potential investors. They just need a system
that’s available one or two hours a day for
demos. They aren’t looking for a bulletproof
24x7 production server. In fact, they can’t afford
to PAY for a bulletproof system right now.
They’d rather have more features to show off,
even if they might not handle a high level of
throughput. It would probably take significantly
more time and /or resources to produce a system
with guaranteed stability. If the customer isn’t
willing to pay the price, they can’t expect to get
it for free.
In XP, the customer’s role is to make business
decisions, not to be a quality expert. Face it,
some people are always on the “happy path”...
just as my husband and I were when we signed a
contract with our builder for our home addition.
As the tester, I feel it’s my responsibility to help
the customer make conscious decisions about
quality during the planning process. If the
customer is clear about his acceptance criteria,
and these are reflected accurately in the
acceptance tests, we’re much more likely to
achieve the level of quality the customer wants,
without giving our time away for free.
Internal and External Quality
In Extreme Programming Explained, Kent Beck
“There is a strange relationship
between internal and external quality.
External quality is quality as measured
by the customer. Internal quality is
quality as measured by the
He goes on to explain the human effect on
“If you deliberately downgrade quality,
your team might go faster at first, but
soon the demoralization of producing
crap will overwhelm any gains you
temporarily made from not testing, or
not reviewing, or not sticking to
In this light, it looks as if we should always
strive for the highest standard of quality. This
would of course make me very happy. But is the
customer willing to pay for it?
I think the important concept here is the
difference between internal and external quality.
Whenever I meet someone who works in an XP
environment, they always tell me that one of the
reasons they love coming to work each morning
is they know they’ll be allowed to do their best
work. If you take that away, XP won’t work.
It’s good to have 100% successful unit tests. In
the long run, it speeds up development time.
Internal quality should be a given.
External quality can be defined as a set of
features, for example:
• Whenever the user makes a mistake, a userfriendly
error screen appears
• It’s impossible to crash the server via the
• The system can handle a hundred concurrent
• The system will stay up 99.995% of the time
Negotiating with the customer on external
quality doesn’t mean skimping on acceptance
tests or deliberately producing unstable code. It
means that the customer asks for a certain
standard of quality and pays for it. If they want a
system to handle all exceptions, that should be in
the story – or multiple stories. Story one says to
implement this functionality; story two says to
make the functionality work with N concurrent
users hammering it.
The XP Tester as Quality Assurance Engineer
The XP books say that the customer writes the
test. In Extreme Programming Explained, Kent
Beck says customers need to ask themselves,
“What would have to be checked before I would
be confident this story was done?” This very
question implies tests that check for intended
functionality, or what my boss calls “Happy
... to read more articles, visit http://sqa.fyicenter.com/art/