Writing Software Security Test Cases
By: Robert Auger
Part of software testing involves replicating customer use cases against a given application. These use cases are
documented in a test plan during the quality assurance phase in the development cycle to act as a checklist ensuring
common use cases aren't missed during the testing phase. People within the quality assurance community are
starting to understand that checking an application for security issues (defects) isn't just the responsibility of the
security department (if one exists), or the software architects. While typical QA Engineers don't understand the scope
or inner working of specific software vulnerabilities, they do go about testing an application in a similar fashion to how
the penetration testing community does. Unlike typical penetration testing QA has access to
internal documents and insider information giving them advantages to aide in the testing of an application. In addition
to documenting customer use cases it's important to begin the process of documenting what an attacker may attempt against
your application as well and incorporating these attacker 'use cases' into a security section of your standard test plan.
In Preparation For Your Test Plan
Before writing a test case it is important to identify the scope of your testing. I've broken up
prep work into three sections below.
Identify Application Input
- Files
- Environmental Variables
- Parameters (This may be received by a web application, command line argument, application form textbox or control, etc...)
- Configuration Files
- The Windows Registry
- Database Source (SQL Database, LDAP, etc...)
- Hidden/Debugging Commands
- Other (Ask your development team for anything else taking in input)
Identify every vector that input comes from that could possibly be modified by a user. Perform basic bounds testing along with security
related input validation tests. The
document written by contains
a detailed list of the types of vulnerabilities caused by modifying input along with examples. If an application takes a value by the user
and performs a database/ldap query check for issues such as
and . You
may have heard of ,
or .
If you enter in a large amount of data into an application input and the application errors, crashes, loops, acts funny, or shuts down this may be a sign
of a potential buffer overflow. Your application developer should be able to track down the cause of the crash and identify the exact cause. Simply
put when a large or poorly formatted input crashes an application there is cause for concern from both the product stability perspective
as well as security.
Data may travel various paths and each path may be a vector for an attacker to exploit the application. Your development team should
be able to inform you of how the application input is used, where it goes, and provide suggestions on how to test for security, and non
security related issues.
Identify Application Output
- Files
- Environmental Variables
- Network Traffic
- The Windows Registry
- Console/Form
- Database Source
- Hidden
- Other
Identify every way that the application outputs data. This data doesn't have to be text displayed on the screen, it could be binary data
in the form of a file, network stream, or other. Once again I'm going to refer to the document for common types of output vulnerabilities. One of
the most common is a
vulnerability type whose impact is only starting to become more apparent. Originally thought to only affect web based applications it is
starting to in .
Take into consideration that data outputted to a file may
come from a location modifiable by the user and during the input phase is properly sanitized and appear harmless, however could
cause damage to other portions of your application trying to read this file later on. , , , , and even
have been found to contain security flaws related
to processing of data.
Installation and Deployment
- Installation
- Using an installer
- Using instructions
- Using a zip file or other archive
- Deployment
- Trusted Environment
- Untrusted Environment/Third Party shared hosting environment
Installed applications must take into consideration of the permissions being applied to files and registry keys during installation
and during the applications execution. Even temporary files existing for no longer than 1 second during installation or during the applications execution
could be all an attacker needs to gain access to sensitive application data or a user's machine. Unix machines have the concept of a user mask
which defines how a newly created file's permissions are set. During deployment or installation consider that the user may have a poorly defined
user mask which could leave your application open to attack. Ensure that any installer or deployment tool sets permissions appropriately for
files, newly created databases, and registry keys (when applicable). If you provide instructions to the user for how to set up your application
review the steps involved, permissions suggested and the possible impact of this. One common mistake in relation to web based applications involves
telling the user to chmod a certain file or directory to 777. More than a handful of outline the risks of this permission.
Sample Security Test Cases For A Shopping Cart Application
Below I've split up security test cases into Functional and Logical. The look and feel of your own Test plan format may vary however you should get
the point. This is not an extensive sample of what a shopping cart test plan may look like but should convey what sorts of tests cases one might add as well
as how uniform many of them are.
Functional Tests
Function Category
- Specific Test Case
Customer Order File
* Ensure that 'orders.txt' file permissions are as restrictive as possible. If these permissions are loosely defined then this as a severity 1
security issue.
* Ensure that sensitive data within the 'orders.txt' file is encrypted using a known strong algorithm. This is a severity 1 security issue.
Customer Data Stored in a SQL Database
* Ensure that sensitive data within the SQL Database is encrypted using a known strong algorithm. This is a severity 1 security issue.
Registration Form
* For each user input perform common security related input validation tests. See for a list of
common input vulnerability types. For each input perform each vulnerability type. The severity level of a vulnerability will be determined by the
vulnerability type, and probability.
* (If SQL is Used) Perform both standard SQL Injection, and Blind SQL Injection tests as outlined by and . If SQL
Injection is present file this as a severity 1 issue.
Login
* For each user input perform common security related input validation tests. See for a list of
common input vulnerability types. For each input perform each vulnerability type. The severity level of a vulnerability will be determined by the
vulnerability type, and probability.
* (If SQL is Used) Perform both standard SQL Injection, and Blind SQL Injection tests as outlined by and . If SQL
Injection is present file this as a severity 1 issue.
Buying Items
* Ensure that the user is unable to modify the price for a given item. Ensure that the price is not exposed in a web form,
cookie, query string, or POST data. If the price is exposed through one of these vectors ensure that if changed, the
application detects the modification on the server side and refuses to sell
the item for anything other than the stated price.
* For each user input perform common security related input validation tests. See for a list of
common input vulnerability types. For each input perform each vulnerability type.
Search Engine
* For each user input perform common security related input validation tests. See for a list of
common input vulnerability types. For each input perform each vulnerability type.
* (If user text is echo'd back) Test for vulnerabilities. If discovered file a severity 2 issue.
Logical Tests
Authentication
* Disallow the user from directly accessing a page requiring the user to be logged in. If you are able to access a page requiring authentication
and are currently not logged in, file a security ticket of severity 2-3 depending on the page.
Login
* If the username or password is incorrect do not display a message such as 'Your Username is Incorrect'. Instead ensure that a generic message
stating 'Invalid Credentials' is displayed so as not to leak information about existing user accounts. If either the username or password is specifically
mentioned file as a severity 3 issue.
Email Confirmation
* When emailing the order confirmation do not leak sensitive information such as the full credit card number, social security number,
full name and address, or other similar sensitive information. If the card number or social security referenced in the email displays more than
the last 4 digits (Unless otherwise defined) file a severity 2 issue.
Password Reset
* Ensure that if the password is reset that the temporary password expires after a reasonable amount of time as defined by the product manager (default is 48
hours). After 48 hours the temporary password will expire and will need to be regenerated.
Full article...
Other Resource
... to read more articles, visit http://sqa.fyicenter.com/art/
|