Quality Practices and Problems in Free Software Projects
By: Martin Michlmayr, Francis Hunt, David Probert
Abstract—Free software and open source projects are often
perceived to be of high quality. It has been suggested that the
high level of quality found in some free software projects is
related to the open development model which promotes peer
review. While the quality of some free software projects is comparable
to, if not better than, that of closed source software, not
all free software projects are successful and of high quality.
Even mature and successful projects face quality problems;
some of these are related to the unique characteristics of
free software and open source as a distributed development
model led primarily by volunteers. In exploratory interviews
performed with free software and open source developers,
several common quality practices as well as actual quality
problems have been identified. The results of these interviews
are presented in this paper in order to take stock of the
current status of quality in free software projects and to act
as a starting point for the implementation of quality process
improvement strategies.
Index Terms—quality practices, quality assurance, quality
improvement, free software, open source.
I. INTRODUCTION
In recent years, the free software and open source model
has established itself as a viable alternative to other development
models. Many popular products, such as Linux,
Apache and Samba, have been created according to open
source practices, and the number of free software applications
is increasing steadily. An empirical study of a
large collection of free software conducted in 2001 found
more than 55,000,000 lines of code, with an estimated
value of almost US$1.9 billion according to the COCOMO
model [4]; moreover, the system has more than doubled in
size since then [5].
of significant pieces of software, but many of these applications
show levels of quality comparable to or exceeding
that of software developed in a closed and proprietary
fashion [6], [14]. In his paper “The Cathedral and the
Bazaar”, Raymond suggests that this high level of quality
is partly due to the high degree of peer review and user
involvement often found in free software projects [13]. This
hypothesis is in line with findings from traditional software
engineering research, which have shown that peer review
has a significant impact on the quality of software [3].
While Raymond’s hypothesis sounds plausible, his model
has yet to be validated and tested rigorously through empirical
data analysis. In general, much of what has been
said about quality in free software is based on anecdotal
rather than empirical evidence. Given the recent success
of many free software applications it can be assumed that,
to some extent, the software fulfils the requirements and
quality standards of users. Nevertheless, empirical work is
necessary to study quality in free software and to contrast
it with that of proprietary software. In particular, work on
the identification of problems related to the free software
model would be beneficial. Insights from such research
can subsequently be used to propose process improvement
strategies.
II. BACKGROUND
Free software projects usually exhibit two characteristics
that have important implications on quality assurance. First,
they are distributed, in many cases all over the world.
Second, the participants of free software projects are usually
unpaid volunteers. The recent interest in free software,
along with its deployment, clearly shows that successful and
mature products for end-users can be built by volunteers
in a distributed fashion. The free software development
model offers certain advantages over proprietary models;
these include the potential for substantial peer review and
the attraction of excellent programmers from across the
world. However, free software also faces certain challenges
that are unique to this model. For example, due to the
voluntary nature of free software projects, it is impossible to
fully rely on project participants [11]. This issue is further
complicated by the distributed nature because it makes
it difficult to identify volunteers who are neglecting their
duties, and to decide where more resources are needed [10].
While most research on open source has focused on
popular and successful projects such as Apache [12] and
GNOME [9], there is an increasing awareness that not
all free software projects reach maturity, success or high
quality. SourceForge, which is currently the most popular
hosting site for free software and open source projects with
over 95,000 projects, is not only a good resource to find
well maintained free software applications – there are also
a large number of abandoned projects and software with
low quality [7]. Some of these abandoned projects may be
explained in terms of a selection process given that more
interesting projects with a higher potential will probably
attract a larger number of volunteers, but it has also been
suggested that project failures might be related to the lack
of project management skills [15]. Nevertheless, large and
successful projects also face important problems related to
quality, such as the volatility of volunteer contributions [11],
[10] and the inability of developers to keep up with bug
reports [17].
In order to ensure that free software remains a viable
model for the creation of mature and high quality software
suitable for corporate and mission-critical use, free software
quality assurance and project management have to take these
challenges and other quality problems into account and
find solutions to them. As a first step in the development
of practices and processes to assure and further improve
quality in free software projects, existing quality practices
and quality problems have to be clearly identified. To date
however, only a few initial surveys on quality related activities
in free software projects have been conducted [19], [20].
The rest of this paper will present results from exploratory
interviews conducted with free software developers from a
diverse range of projects. The insights gained from these
interviews lead to a more complete account of the current
status of quality practices and problems.
III. METHODOLOGY
For this study, exploratory interviews with seven free
software and open source developers have been conducted.
The projects were of a very diverse nature and it is believed
that despite the relatively small sample size a fair
representation of various kinds of free software projects is
given. One project started at a university was aimed at giving
undergraduate students the opportunity to get experience of
a software development project. This project was later transformed
into a community project in which students of this
university participated with external contributors. Another
project was part of research activities carried out by a large
corporation, and one resulted from a university research
project. The remaining projects originated as community
projects. The size of the projects varied quite widely, from
large projects such as Apache, GNOME and Debian, to
smaller efforts such as VideoLAN and Dasher.
The interviews were of a fairly unstructured nature so as
to allow a thorough exploration of quality in the projects
in which the interviewees were involved. Questions, such
as the following, were asked directly but the interviews
remained open for the exploration of other topics:
• Are there any quality issues in your project? If so, what
are they and how do you deal with them?
• What techniques can be applied in free software and
open source projects to ensure quality?
• What kind of facilities does your project have to ensure
quality?
• How would you compare the quality of open source
and proprietary software? When might the quality in
one be higher than in the other?
Full article...
Other Resource
... to read more articles, visit http://sqa.fyicenter.com/art/
|