Software QA FYI - SQAFYI

Using Fuzzers in Software Testing

By: Robert Auger

Using Fuzzers in Software Testing

The security industry has been using Fuzzers to identify security vulnerabilities in popular products for years. This immediately tells us two things

1) The vendors of the vulnerable products aren't using fuzzers, or using them properly
2) Fuzzers can yield results

Fuzzers are used to perform negative testing against application inputs to identify unexpected behaviors. This is accomplished by throwing valid and invalid data in application inputs. Fuzzers can be good tools to identify filtering problems (E.g. a email address verifier that does a poor job) and after initial setup require little maintenance with the exception of medium to major application changes. These behaviors may be harmless bugs, but could lead to some sort of security risk. A hacker/security person typically will identify an application's inputs and outputs, and adjust a fuzzer application to start throwing bad data at them. Depending on the fuzz session configuration they can run for hours, days, weeks, even months. A software tester (QA) can also utilize this common method to identify the same sorts of issues, and incorporate it into their reguler testing cycle. For the scope of security testing the primary focus lies in fuzzing input fields to perform negative testing.

The goal is to make the application panic, error, act strangely, and/or possibly terminate. Depending on the application these errors may result in the discovery of a buffer overflow issue, poor application error handling, format string vulnerability, sql injection flaw, or other security related problem. So what should you be fuzzing exactly? All identifiable application inputs and outputs. This could be against a console or web based application. For application output take into consideration how that output is used at a later time during execution, and also consider how it may tie into other related applications. For example sticking a large amount of data into a database of some sort may cause that application to have issues, which may impact the overall flow of your application. Below are the most common locations that you'll fuzz in an application.

Common Input and Output Locations
- Environmental Variables
- Registry keys
- Files
- Network Streams
- Databases

To ensure all application inputs/output paths have been discovered, it is important to work with your development team to ensure nothing is missed. Also take into consideration your applications various configurations, for example an application supporting both MSQL or Oracle.

Selecting a fuzzer

There are plenty of freeware fuzzers available, each focused towards a particular area. If you find yourself unable to find a fuzzer to fuzz your type of application, consider utilizing a fuzzing framework like 'Peach Fuzzer Framework' which is available in C++/Java/.NET and has a good tutorial available to build your own harnesses. Below is a list of popular fuzzers by common focus.

Network Traffic:
Scratch
DFuz

HTTP
SPIKE

File Fuzzing
Mistress
FuzzyFiles
FileFuzz, ComRaider, SPIKEFILE, and NOTSPIKEFILE

The Fuzzing Mailing list is a good place to start to see how others configure, utilize, and develop fuzzers. Incorporating fuzzers into your QA test plan and cycle is an additional step that can be used to identify potential security issues as well as regression of a particular issue. The act of automating negative testing isn't a new idea and by utilizing a fuzzer you are more easily automating and expanding upon this essential part of software testing.

Additional Reading and References

Fuzzing Mailing List
Fuzzing Tools List
http://en.wikipedia.org/wiki/Fuzz_testing


Other Resource

... to read more articles, visit http://sqa.fyicenter.com/art/

Using Fuzzers in Software Testing