Context Driven Testing
By: by KentBeck
What is Context-Driven Testing?
The context-driven school of software testing advocates continuous and creative evaluation of testing opportunities in light of the potential information revealed and the value of that information to the organization right now. The context-driven school of software testing advocates testing in a way that conforms to the context of the project, as opposed to testing in a way that follows some fixed notion of "best practice". It declares that the human part of that context is most important, and that good testing is ultimately a matter of skill, not procedure.
KentBeck has criticized BobBinder's writing on software testing because it only focusses on benefits without regard to costs. Cost-benefit analysis is central to context-driven testing.
Context-driven testing could arguably be called agile testing because the principles it recommends are analogous to those suggested in the AgileManifesto. Specifically, context-driven testers would answer each of the following questions in the affirmative:
1. Do we put more value in individuals and their interactions over processes or tools?
2. Do we put more value in seeing working software over documentation?
3. Do we put more value in collaborating with our customers (including development, which context-driven testers see as an important customer of testing services) over negotiating contracts (and specifications).
4. Do we put more value in responding to change over following the plan?
However, no context-driven tester would want to be limited to the few testing techniques recommended by specific agile methodologies like ExtremeProgramming. (See discussion below for more on this point.)
Context-driven software testing is a set of values about test methodology. It is not itself a test technique. To be a context-driven tester is to approach each testing situation as if it were unique in important ways, and to develop the skills to react to situations with a broad and deep awareness of problems in projects and possible testing-related solutions to those problems.
The Seven Basic Principles of the Context-Driven School
1. The value of any practice depends on its context.
2. There are good practices in context, but there are no best practices.
3. People, working together, are the most important part of any project's context.
4. Projects unfold over time in ways that are often not predictable.
5. The product is a solution. If the problem isn't solved, the product doesn't work.
6. Good software testing is a challenging intellectual process.
7. Only through judgment and skill, exercised cooperatively throughout the entire project, are we able to do the right things at the right times to effectively test our products.
The Development of the Context-Driven School
The context-driven school was founded by CemKaner, BrianMarick, JamesBach, and BretPettichord. The school was founded on November 21, 1999, when CemKaner, JamesBach, and BrianMarick were discussing the possibility of writing a testing book together. A few days later, they established the context-driven testing mailing list on Yahoogroups (software-testing). Two years later, the first self-described context-driven testing book (Lessons Learned in Software Testing: A Context-Driven Approach) was published by Cem, James, and BretPettichord. The context-driven testing manifesto was also distributed at that time.
Although the school was declared in 1999, its direct roots can be traced back to the eighties, when James and Cem were independently trying and failing to make prevailing "best practices" of testing work in the world of commercial mass-market software products. Independently, they developed test methodologies that were agile and practitioner-centered. In 1995, James and Cem met and began to collaborate with each other and with like-minded people to find better ways of analyzing and discussing the relative value of technical practices.
CemKaner and BrianLawrence? began hosting the Los Altos Workshops on Software Testing in 1997. These brought many testers together to share experiences and learn how to share ideas across barriers of project size, market, technology, and many other aspects of project context. In many ways, this was really the beginning of school of context-driven testing. Early participants in LAWST workshops included JamesBach, BrianMarick, BretPettichord, DougHoffman?, ElisabethHendrickson, NoelNyman?, JohannaRothman? ...
Many context-driven testers have looked to philsophy of science and epistemology for insight into how to think about software testing. One example of this is interpreting KarlPopper 's verification principle for testing scientific theories as a way of thinking about testing software.
Early Context-Driven Writings
In the years between 1992 and 1999 show a development of the ideas of context-driven testing in various writings.
CemKaner published Testing Computer Software and said that his was a book for when "the programmers don't, won't, and don't have to follow the rules." Prior to its publication, most testing literature said that testers could only be effective if they programmers followed the waterfall method, providing detailed specifications. This book was welcomed by testers who had worked with high-performance teams that had succeeded because they focused on speed and working code over extensive planning and documentation.
JamesBach wrote "Process Evolution in a Mad World" in '93, "The Immaturity of the CMM" in '94, and a series of other articles in '97 and '98 as part of his Software Realities column in IEEE Computer. "Good Practice Hunting", written in '99, was published shortly before the school was declared, and was inspired by the discussions at LAWST. All these articles may be found at http://www.satisfice.com. Many of these articles were seen as defining the validity of an attitude towards development that later became labelled "agile."
BrianMarick wrote "Classic Testing Mistakes."
* It is not a testers' job to demand documentation. Good testers work with the information they have, use unofficial documents, and are specific when requesting additional information.
* Testing is a information-providing service, not a "quality assurance" function.
* The value of testing is determined by whether it provides useful and timely information (about the quality of the software).
* There are no guarentees. We don't hide behind the test plan. We learn how to improve testing as we test.
* A tester is a customer advocate. Testers try to understand the customer position and make the best case when they feel it isn't being address.
Testing techniques developed in the Context-Driven Testing School include:
... to read more articles, visit http://sqa.fyicenter.com/art/