Software QA FYI - SQAFYI

Bug writing guidelines

By: Mozilla

Bug writing guidelines

The Mozilla bug tracking system (Bugzilla) allows any interested individuals on the Internet to directly report and track bugs in mozilla.org open-source projects like the Mozilla Application Suite or Mozilla Firefox.

Like you, Mozilla QA (Quality Assurance) wants your bug reports to result in bug fixes; the more effectively a bug is reported, the more likely that an engineer will actually fix it.

By following these guidelines, you can help ensure that your bugs stay at the top of the Mozilla engineers' heap, and get fixed. Bugzilla is the preferred method of submitting a bug -- the linked entry form incorporates parts of these guidelines.

How to enter your useful bug report into Bugzilla

Before you enter your bug, you need to make sure it has not been previously reported. There is a tutorial on the best ways of doing this.

Next, be sure that you've reproduced your bug using a build released within the past three days. Our development process moves at lightning speed, and the bug you've found may already have been fixed. (Nightly builds can be downloaded from ftp.mozilla.org, see [1] for specific links.)

If you've discovered a new bug using a current build, report it in the guided Bugzilla entry form.

  1. Are you sure you don't want to use the guided form? You won't have to read the rest of this page if you do.
  2. Okay, then. From the Bugzilla main page (https://bugzilla.mozilla.org), choose "Enter a new bug".
  3. Select the product that you've found a bug in.
  4. If you haven't logged into Bugzilla already, you'll need to enter your email address and password, then press the "Login" button. (If you don't yet have a password, enter your email address below and press the "Submit Request" button instead. You'll receive an email message with your password shortly.)

Now, fill out the form. Here's what it all means:

Where did you find the bug?

Product: In which product did you find the bug?
You just filled this out on the last page.
Version: In which product version did you find the bug?
We're not yet using this field. Just leave the default value as you found it. ;)
Component: In which component does the bug exist?
Bugzilla requires that you select a component to enter a bug. (If they all look meaningless, click on the Component link, which links to descriptions of each component, to help you make the best choice.)
Platform: On which hardware platform did you find this bug? (e.g. Macintosh, SGI, Sun, PC.)
If you know the bug happens on all hardware platforms, choose 'All'. Otherwise, select the platform that you found the bug on, or "Other" if your platform isn't listed.
OS: On which Operating System (OS) did you find this bug? (e.g. Linux, Windows NT, Mac OS X)
If you know the bug happens on all OSs, choose 'All'. Otherwise, select the OS that you found the bug on, or "Other" if your OS isn't listed.

How important is the bug?

Severity: How damaging is the bug?
This item defaults to "normal". (To determine the most appropriate severity for a particular bug, click on the Severity link for a full explanation of each choice, from Critical to Enhancement.)

Who will be following up on the bug?

Assigned To: Which engineer should be responsible for fixing this bug?
Bugzilla will automatically assign the bug to a default engineer based on the component when you submit the bug report; this text box lets you manually assign it to a different engineer. (To see the list of default engineers for each component, click on the Component link.)
Cc: Who else should receive e-mail updates on changes to this bug?
List the full e-mail addresses of other individuals who should receive an e-mail update upon every change to the bug report. You can enter as many e-mail addresses as you'd like; e-mail addresses must be separated by commas, with no spaces between the addresses.
You would not normally change either of these fields from their default values.

What else can you tell the engineer about the bug?

URL: On what URL did you discover this bug?
If you encountered the bug on a particular URL, please provide it (or, them) here. If you've isolated the bug to a specific HTML snippet, please also provide a URL for that, too or, preferably, return to the bug after you've submitted it and add the HTML snippet as an attachment.
Summary: How would you describe the bug, in approximately 60 or fewer characters?
A good summary should quickly and uniquely identify a bug report. Otherwise, developers cannot meaningfully query by bug summary, and will often fail to pay attention to your bug report when reviewing a 10 page bug list. Think of it as a "title".
A summary like "Drag-scrolling any web page crashes Mac OS X builds" is a useful title. "Crash" or "Drag Crash" would be examples of a bad title.

Bug description

Description: What else can you tell the engineer about this bug?
Please provide as detailed of a problem diagnosis in this field as possible, using the following example as a template to go by:

Overview Description: More detailed expansion of summary.

Drag-selecting any page crashes Mac OS X builds in NSGetFactory

Steps to Reproduce: The minimal set of steps necessary to trigger the bug. Include any special setup steps, and anything else necessary for somebody else to see the bug for themselves.

1) View any web page. (I used the default sample page,
   resource:/res/samples/test0.html)
2) Drag-select the page. (Specifically, while holding
   down the mouse button, drag the mouse pointer downwards
   from any point in the browser's content region to the
   bottom of the browser's content region.)

Actual Results: What the application did after performing the above steps. Include enough information so that somebody testing a build that no longer has the bug can be sure that the bug is fixed without finding a buggy build to test.

The application crashed. Stacktrace appended below from gdb.

Expected Results: What the application should have done, were the bug not present.

The window should scroll downwards. Scrolled content should 
be selected. (Or, at least, the application should not crash.)

Build Date & Platform: Date and platform of the build that you first encountered the bug in. (The build date can be usually found on the about: URL or in the application's About dialog box.)

2003060709 installer build on Mac OS X

Additional Builds and Platforms: Whether or not the bug takes place on other platforms or browsers.

- Occurs On Seamonkey (20030605 build on Windows 2000)
- Doesn't Occur On Seamonkey (20030602 build on SUSE Linux),
  IE 6 (Windows XP), Netscape Navigator 4.5 (Mac OS)

Additional Information: Talkback crash IDs, and any other debugging information that's at most 20 lines long. For crashing bugs:

  • Win32: If you receive a Dr. Watson error, please note the type of the crash, and the module that the application crashed in. (e.g. access violation in mozilla.exe)
  • Mac OS X: When the application crashes, click the "Report" button in the notification window that appears, then copy all the text from the text box under the message "Problem and system information" and include it with your bug report. There's no need to send the bug to Apple, so just click the red close box at the top of the window.
  • Unix: Please provide a minimized stack trace, which can be generated by typing gdb mozilla core into a shell prompt.
Date/Time:      2006-12-26 12:15:20.089 -0500
OS Version:     10.4.8 (Build 8L2127)
Report Version: 4

Command: firefox-bin
Path:    /Applications/Firefox.app/Contents/MacOS/firefox-bin
Parent:  WindowServer [71]

Version: 2.0.0.1 (2.0.0.1)

PID:    114
Thread: 0

Exception:  EXC_BAD_ACCESS (0x0001)
Codes:      KERN_PROTECTION_FAILURE (0x0002) at 0x000000ca

Thread 0 Crashed:
0   libxpcom_core.dylib 	0x0186329b AppendUTF8toUTF16(char const*, nsAString_internal&) + 31
1   libxpcom_core.dylib 	0x01822916 nsTextFormatter::smprintf_free(unsigned short*) + 3248
... (many many more lines like this) ...

Add an attachment: After you submit the bug, you will have a chance to add attachments (additional files) to the bug. If you have an HTML file that shows the bug (without other files needed), you should attach it to the bug so that anybody looking at the bug can click on it to view the file. If you need to attach more than one file (such as an image and an HTML file), attach the image first, and then edit the HTML file to point to the URL of the attached image. But don't attach more than a few files.

Also, if you have debugging information that is more than 20 lines long, it should be an attachment rather than pasted into the Additional information.

You're done!

After double-checking your entries for any possible errors, press the "Commit" button, and your bug report will be part of the Bugzilla database. Once it's there, don't forget to add any attachments (testcases, debugging information) that you need to add.


Other Resource

... to read more articles, visit http://sqa.fyicenter.com/art/

Bug writing guidelines