Software QA FYI - SQAFYI

Testing Dirty Systems

By: Randall W. Rice

What is a “Dirty System?”
Dirty systems are defined as systems for which the tester may lack any of the following:

1. Complete documentation;
2. Full understanding of requirements;
3. Design/Architecture is not done in a standardized format; and/or
4. The defect density of the software is unknown.

Dirty Systems Can Be Tested
• Traditional back-end testing approaches generally do not work with dirty systems.

• A life-cycle or “filtering” approach is needed if testing is going to be effective.
• However, these are merely extensions of good testing practices.
• Test planning becomes like being a “software archeologist” – trying to discover what the software does by shifting through mounds of debris.

Why Is It Important That Testing Dirty Systems Uses A Process?

• Two distinct tasks to perform in order to conduct testing
– Determine how the test will be conducted (about 1/3 of the testing effort)
– Actually conduct the test (the remaining 2/3 of the effort)
• This time expenditure allocation is based on having a test planning process in place
• How you do the first 1/3 will greatly affect the other 2/3 of the effort!

The “Black Holes” of System Knowledge

• Understanding overly complex logic
• Inability to relate software functionality to defined requirements and objectives
• Absence of defined requirements
• Incorrect or vague requirements
• Missing, incomplete, or obsolete systems documentation
• Large size and scope of functionality
• Vendor-developed software that is no longer supported by the vendor
• Inadequate user training
• Lack of planning to accommodate growth and technical integration issues
• Inadequate security policy to protect software and data

Dirty System Test Planning

• Objective – To define in advance what is to be tested and how to evaluate the test results.
– A characteristic of dirty systems is the lack of an initial rigorous test, and therefore, an initially inadequate test plan. It is not uncommon in legacy systems to develop a comprehensive test plan years after the system is implemented

Dirty System Test Execution • Objective – To perform the planned test and to evaluate the test results.

– Much like an exterminator knows where to find certain kinds of pests due to the knowledge of where they thrive, you can also become an expert software bug exterminator by identifying common breeding ground for categories of software bugs

Challenges of Dirty Systems Test Analysis

• Knowing when testing is reasonably complete
• Knowing if test results are correct
• Dealing with frequent, and sometimes rapid, changes to dirty systems
• Building and maintaining a baseline set of expected test results
• Dealing with the human factors
– Mismanagement and staff turnover, which impacts the knowledgebase for a system
• Lack of a way to observe the test results
• Lack of a robust environment to conduct a reliable test

Full article...

Other Resource

... to read more articles, visit

Testing Dirty Systems