Testing Dirty Systems
By: Randall W. Rice
What is a “Dirty System?”
Dirty systems are defined as systems for which
the tester may lack any of the following:
1. Complete documentation;
2. Full understanding of requirements;
3. Design/Architecture is not done in a
standardized format; and/or
4. The defect density of the software is
unknown.
Dirty Systems Can Be Tested
• Traditional back-end testing approaches
generally do not work with dirty systems.
• A life-cycle or “filtering” approach is
needed if testing is going to be effective.
• However, these are merely extensions of
good testing practices.
• Test planning becomes like being a
“software archeologist” – trying to discover
what the software does by shifting through
mounds of debris.
Why Is It Important That Testing
Dirty Systems Uses A Process?
• Two distinct tasks to perform in order to
conduct testing
– Determine how the test will be conducted (about
1/3 of the testing effort)
– Actually conduct the test (the remaining 2/3 of the
effort)
• This time expenditure allocation is based on
having a test planning process in place
• How you do the first 1/3 will greatly affect
the other 2/3 of the effort!
The “Black Holes” of System Knowledge
• Understanding overly complex logic
• Inability to relate software functionality to defined
requirements and objectives
• Absence of defined requirements
• Incorrect or vague requirements
• Missing, incomplete, or obsolete systems
documentation
• Large size and scope of functionality
• Vendor-developed software that is no longer
supported by the vendor
• Inadequate user training
• Lack of planning to accommodate growth and
technical integration issues
• Inadequate security policy to protect software and
data
Dirty System Test Planning
• Objective – To define in advance what
is to be tested and how to evaluate the
test results.
– A characteristic of dirty systems is the lack
of an initial rigorous test, and therefore, an
initially inadequate test plan. It is not
uncommon in legacy systems to develop a
comprehensive test plan years after the
system is implemented
Dirty System Test Execution
• Objective – To perform the planned test
and to evaluate the test results.
– Much like an exterminator knows where to
find certain kinds of pests due to the
knowledge of where they thrive, you can
also become an expert software bug
exterminator by identifying common
breeding ground for categories of software
bugs
Challenges of Dirty Systems
Test Analysis
• Knowing when testing is reasonably complete
• Knowing if test results are correct
• Dealing with frequent, and sometimes rapid, changes
to dirty systems
• Building and maintaining a baseline set of expected
test results
• Dealing with the human factors
– Mismanagement and staff turnover, which impacts the
knowledgebase for a system
• Lack of a way to observe the test results
• Lack of a robust environment to conduct a reliable
test
Full article...
Other Resource
... to read more articles, visit http://sqa.fyicenter.com/art/
|