Pair Testing: How I Brought Developers into the Test Lab
By: Jonathan Kohl
AS A BLACK BOX TESTER WORKING ON A
variety of development projects, I was often
approached by developers who were
trying out test-driven development and
wanted to learn more about testing. To
show them how I worked. I involved
them in pair testing, a technique in which
two people test an application at the
same computer. Interestingly enough, the
developers taught me as much as I taught
them.
Let me explain how it worked. Before
a pair testing session, the developer and I
would meet to determine the focus and
scope of the test. We would pick an area
of the program to test and establish our
goal. Sometimes the goal would be to
track down an elusive bug, sometimes to
ensure that customer acceptance test criteria
were met, or sometimes to find bugs
in a new piece of functionality. We would
write our goals and testing ideas on a
whiteboard, keeping a copy of the finished
notes for our own use.
Later, the developer would join me at
the computer. He would watch how my
exploratory testing techniques led to new
ideas and plans of attack. Soon, he too
would be actively testing. As the sessions
progressed, we would revisit our goals
and come up with new ideas for testing
Mutual Learning
The benefits of pair testing extend beyond
the developer and tester involved. Developers
learn how to test
their own code more effectively
and gain a new
perspective on how their
software might be used.
Testers gain a more thorough
understanding of
the application they are
testing and learn debugging
techniques to find
causes of defects. Both developers
and testers learn how to uncover more information to
write more effective defect reports. Additionally,
pair testing can break down communication
barriers between developers
and testers and facilitate team building.
During pair testing sessions, developers
regularly told me that they had never
thought of looking at software testing the
way I did. The “what would happen if I
tried this…” mindset was not as intuitive
to them. They tended to
think of tests that reflected
a typical way of using
the software; I thought of
tests that might cause
failures. As I applied my
usual techniques they
typically would say
something such as,
“Stop! Slow down! What
are you doing?” Then I
would stop and explain my reasoning. The types
of bounds conditions or
input validation tests that
I immediately tried on input
fields of new features
surprised them. They
were also surprised by my
attempts to get around
the control flow of the
program.
The developers weren’t
the only ones learning
something new. I was often
struck by how developers’
perspectives and
testing techniques were
different from my own. I
tend to focus on the user’s
perspective, while developers
tend to focus on
what the program tells
them about what is going
on behind the scenes. As
they shared their knowledge
of the underlying
code, I learned what areas of the product
might contain weaknesses and gained application-
specific knowledge that helped
me track down problems more effectively
and provided more information when reporting
a defect.
Once developers applied what they
learned about testing to their development
projects, they came up with great
testing scenarios on their own. Best of all,
it became difficult to find defects in the
developer’s code after pair testing.
Collaborative Problem Solving
Pair testing can also be used as a way to
track down hard-to-reproduce defects.
Once, after several hours of testing an
application, I uncovered a defect that occurred
only sporadically. I suspected
clients had reported this defect in the
form of several seemingly unrelated is sues. To track it down, I pair tested with
a senior developer. While he had a suspicion
of what was causing the defect, we
could not repeat it when testing together.
At the end of that session, he installed a
debugger on my machine and told me
what information needed to be captured
if it occurred again.
The next time the defect occurred, we
tested together until we were confident we
had narrowed down the cause of the
problem. He knew what code was problematic,
so we brainstormed related integration
tests to narrow down where the
problem might occur on the code level.
From that list, I derived several more tests.
In almost every case, the defect occurred.
We were able to track down the problem,
fix it, and develop some solid unit tests for
the fix based on our sample of test cases.
The whole process took only a couple
of days. In my experience as a tester,
tracking down difficult defects on my
own has taken much longer. Through
collaboration, we found the cause more
quickly and were able to ship the software
with a high level of confidence.
Full article...
Other Resource
... to read more articles, visit http://sqa.fyicenter.com/art/
|