Diagnosing evolution in test-infected code
By: Christian Wege,Martin Lippert
ABSTRACT
In this study we trace the effects of applying the
techniques of refactoring and aggressive unit testing in
source code based on historical information. We show
how their impact on the evolution of the architecture can
be testified. The study comprises the analysis of a large
number of indiv idual integration versions of a large
framework. The method described here can help
development teams find weaknesses in their application
of the two traced techniques.
INTRODUCTION
In a world of constantly changing requirements, systems
development must ensure that it is able to quickly
respond to changed user requirements or technology
updates. One major promise of Extreme
Programming(XP) [2] is to enable the construction of an
evolvable system. Instead of planning all possible future
enhancements from the very beginning an extreme
programmer relies on its ability to incorporate changes to
the system and its architecture at an arbitrary point in the
future.
In this study we investigated the artifacts (namely the
source code and other historical information) of a project
which uses the aggressive unit testing and refactoring
techniques extensively for the development. We trace the
effects of the application of these two techniques in the
developed source code. Our system under investigation is
JWAM1 – a framework for constructing large scale
interactive software systems.
In their well-known article Beck and Gamma introduce
the testing style of test infection [1]. For every class in the
system you write a unit test. New requirements are
implemented in the system by refactoring the unit tests
first and then the system classes [4]. So when those two
techniques are applied strictly we talk about test infected
code. Given this definition JWAM is test-infected. Its test
suite created with the Java testing framework provided by
Beck and Gamma2.
The JWAM development relies on an integration server
[7] which ensures that for every update of the source code
all tests still run. The study is based on 254 individual
integration versions of the framework which stem from
this continuous integration process. In addition to the
source code we used the integration log which contains a
small description for every update of the source tree.
Lippert et al. state that “With Pair Programming we have
improved framework quality, with test cases we maintain
it. Without the test cases a lot of the refactoring we did in
the past would have been less smooth.”[8] With the help
of our analysis we validated this rather intuitive statement
by observing the history of specific system properties in
the produced artifacts. As well with the help of our
analysis we can point out some areas of potential
improvements of the framework development.
This study concentrates on tracing the effects of agressive
unit testing and refactoring directly in the code and in
historical information. It doesn’t investigate the
correlation to requirements changes of defect rates, which
would be of high interest as well.
THE CASE STUDY
The method
The method used in our study is an adaptation of the
approach proposed by Mattsson and Bosch [9] for
observing software evolution in object-oriented
frameworks. Based on historical information about the
subsystems, modules and classes they investigated the
size, change rate and growth rate of the system. The work
of Mattsson and Bosch is based on a method proposed in
[5] which focus was on observing the macro-level
software evolution using the version numbering of a
system. Mattsson and Bosch adapted this approach for
investigating object-oriented frameworks. The system
was divided into a number of subsystems which were
themselves divided in several modules. In the adapted
approach each module consisted of several classes
(instead of programs as in the original approach).
Size is calculated by the number of classes in each
module or subsystem. The calculations of change and
growth rate are made in terms of changed classes as units.
Class change is measured in terms of the change in the
number of public methods for each class. The focus on
public methods stems from the fact that a change in the
public methods reflects a better understanding of the
boundary of the system. Changes of private methods
however ma inly reflect refinements of implementation
details and are thus of minor interest.
The method steps in the original approach are3:
1. calculate, for all releases, the change and growth rate
for the whole system,
2. calculate, for all releases, the change and growth rate
for each of the subsystems,
3. for those subsystems that exhibit high growth and
change rates calculate, for all releases, the change and
growth rates for the modules,
4. those modules that exhibit high change and growth
rates are identified as likely candidates for restructuring
[9].
For our study we based our calculations for system,
subsystems and modules on the package/subpackage
structure of Java. The packaging feature of Java is a
natural structuring mechanism provided by the language.
In JWAM this mechanism is used to distinguish between
the core and several non-core part of the whole system
and inside the core to distinguish between the framework
layers. We will discuss this in more detail later. Java
interfaces are treated exactly the same way as Java
classes.
The second important adaptation is that we changed the
top-down approach to a bottom-up approach. Instead of
starting with the top level system, we calculate the values
for every class and subsystem and go up to the top. We
try to trace the development method in the code, therefor
we are interested in all developed artifacts. For being able
to give advice on possible restructuring candidates (like
in the approach by Mattsson and Bosch) we have to
widen the empirical base first.
The third and most important adaptation is the
introduction of the test coverage rate. If aggressive unit
testing is one central part of test-infected programming
then the results should be dependent on the number of
system classes covered by unit tests.
The investigated system
JWAM is a Java framework supporting the development
of large scale interactive software systems according to
the tools & materials approach [11]. The foundation of
the JWAM framework was laid in 1997 by research
assistants and students of the Software Engineering
Group at the University of Hamburg [8]. In 1998 the
commercialization of the framework began. In 1999 the
Full article...
Other Resource
... to read more articles, visit http://sqa.fyicenter.com/art/
|