Software QA FYI - SQAFYI

Using Customer Tests to Drive Development

By: Lisa Crispin

Like many agile software development teams, our team writes tests for each feature before the feature is actually developed. We’ve found many advantages to using tests to drive development, not only at the unit test level but at the functional, system and acceptance test levels. Not only do we have tests which show whether we’ve delivered the correct functionality, but we benefit from increased communication and collaboration, increasing the chances that we will deliver exactly what our customers want. Writing just the right amount of tests and level of detail has proved difficult at times, as has the automation and timing of the automation effort. The effort to overcome those problems has paid off and led us to devote even more resources to driving development with customer tests.

Test-driven development or TDD is a widely accepted practice used by agile software development teams of many flavors – not only Extreme Programming teams. For each small bit of functionality they code, programmers first write unit tests, then they write the code that makes those unit tests pass. TDD is seen as a design tool, since it forces the programmer to think about many aspects of each feature before coding. It results in a ‘safety net’ of tests that can be run with each build, ensuring that new code doesn’t ‘break’ any existing code, or that refactored code maintains its functionality.

Why Customer Test-Driven Development?
I’ve been fortunate to work as a tester on three development teams practicing TDD as they produce J2EE-based web applications. Code produced via TDD far outshines code produced in a ‘traditional’ manner in quality and robustness. Testing code developed in this manner, I don’t find the bugs I was used to finding in ‘typical’ projects, such as bugs produced by boundary conditions or ‘invalid’ input. Don’t get me wrong, as a tester, this makes me happy. So what more do I want?

What I still saw, even with TDD, are misunderstandings between the project’s customers (also known as business experts or product owners) and the software developers. Even if the deployed code was almost bug-free, it didn’t do everything the customer had expected. We can enlarge on the concept of TDD and reduce the risk of delivering the ‘wrong’ code with what I call customer test-driven development.

Customer test-driven development (CTDD), also known as story test-driven development (SDD), consists of driving projects with tests and examples that illustrate the requirements and business rules. What do I mean when I say ‘customer tests’? Basically, anything beyond unit and integration (or contract) testing, which are done by and for the programmers, testing small units of code and their interaction. I use the term ‘customer’ in the XP sense, meaning product owners and people on the business side who specify features to be delivered. Customer tests may include functional, system, end-to-end, performance, load, stress, security, and usability testing, among others. These tests show the customers whether the delivered code meets their expectations.

How CTDD Works
How do we know everything that the customer expects ahead of time? On agile projects, we make this easier by splitting features into small, manageable chunks, known as ‘stories’. Often written on a small index card, stories take a form such as this:
As a retail website customer, I would like to be able to delete items out of my shopping cart so that I can check out with only the items I want.
or each story, we ask the customer to tell us how she will know when this story is complete, in the form of tests and examples. Just as with TDD, before writing any code, we write tests that, when passing, prove the code meets the minimum requirements. These tests are ideally in a form that can be used with an automated tool, but they may also be higher-level tests, or guidelines for later exploratory testing.

Who are these cooperative customers? They’re the people with the domain expertise, who understand business priorities. They may be in sales or marketing, they may be business managers or analysts, they may be end users, they may even be a development manager or customer support. They will most likely need help in writing effective tests that the programmers can use.

Testers provide this help, combining their understanding of the technical requirements of the system with the big picture of what the business needs. Testers ask questions to help elicit requirements and flush out hidden assumptions. For example, with the story above to delete items out of the shopping cart, testers may ask:

Full article...

Other Resource

... to read more articles, visit

Using Customer Tests to Drive Development