background image
<< Elements of a Defect Report | Root Cause Analysis >>

Software Defect Reports

<< Elements of a Defect Report | Root Cause Analysis >>
T e s t R e p o r t i n g a n d M e t r i c s
Software Defect Reports
At a basic level defect reports are very simple:
number of defects by status and severity.
Something like the diagram to the right.
This shows the number of defects and their
status in testing. As the project progresses you
would expect to see the columns marching
across the graph from left to right ­ moving
from New to Open to Fixed to Closed.
More complicated defect reports are of
course possible. For example you might want
to have a report on defect ageing ­ how long
have defects been at a certain status. This
allows you to target defects which have not
progressed, which have not changed status over a
period of time. By looking at the average age of
defects in each status you can predict how long a
given number of defects will take to fix.
By far and away, the best defect report is the defect status trend as a graph. This shows the total
number of defects by status over time (see below).
The great advantage of this
graph is that it allows you to
predict the future.
If you take the graph at
September and plot a line down
the curve of the 'fixed' defects ­
it cuts the x-axis after the end
of December. That means that
from as early as September, it
was possible to predict that not
all of the defects would be fixed
by December, and the project
would not finish on time.
Similarly be following the curve
of the 'new' defects you can
intuit something about the
progress of your project.
If the curve peaks and then is flat, then you have a problem in development ­ the bugs aren't
staying fixed. Either the developers are reintroducing bugs when they attempt to fix them, your
code control is poor or there is some other fundamental issue.
Time to get out the microscope.
34
New
Open Rejected
Fixed Closed
Critical
1
1
0
0
3
Major
4
9
1
8
5
Minor
4
3
5
9
9
Figure 12: Defect status report
Figure 13: Defect trends over time
Jan
Feb
Mar
Apr
May
Jun
Jul
Aug
Sep
Oct
Nov
Dec
0
10
20
30
40
50
60
70
80
90
100
110
120
130
140
Closed
Fixed
Open
New
New
Open
Rejected
Fixed
Closed
0
2
4
6
8
10
12
14
16
18
Defect Status Report
Minor
Major
Critical