Verification and Validation for Trustworthy Software
By: James Bret Michael, Doron Drusinsky, Thomas W. Otani, and Man-Tak Shing
The close interaction between high-integrity systems and their operating environments places a high priority on understanding and satisfying both functional requirements (what the software must do) and safety requirements (what the system must not do). However, traditional validation methods that test the delivered system’s behavior against customer expectations are ineffective (and too late) to assure requirement correctness.
Validating requirements early in the system life cycle is increasingly important to organizations that implement capability-based acquisition. For instance, government organizations such as the US Department of Defense (DoD) now play the role of smart buyers whose job is to acquire a set of capabilities. This makes the task of assuring that the system developers correctly translate capabilities into system specifications even more vital. Without such assurance, the DoD can’t reasonably expect successful development of trustworthy software-intensive systems.
The US Food and Drug Administration (FDA), on the other hand, plays the role of regulator with the responsibility of approving public use of, say, safety critical medical devices and investigating the cause of mishaps involving these devices. The FDA must ensure that the device behaves as the manufacturer specifies and that the manufacturer acts with due diligence in assessing its products’ trustworthiness—without source code or other detailed information about the systems’ implementation.
These examples highlight the need for the continuous and proactive verification and validation (V&V) of complex and safety-critical software systems. This article presents a continuous, computer-aided process that uses statechart assertions, runtime execution monitoring, and scenario-based testing to specify and validate complex system requirements.
Verification & Validation
Merriam-Webster’s Dictionary defines trustworthy as “deserving of trust or confidence; dependable; reliable.” In a system context, dependability is “the trustworthiness of a computer system such that reliance can justifiably be placed on the service;” the dependability of a system can be defined by a set of attributes that include availability, reliability, safety, security (confidentiality and integrity), and maintainability.
The Guide to the Software Engineering Body of Knowledge defines V&V as the process for determining “whether or not products of a given development or maintenance activity conform to the requirement of that activity, and whether the final software product fulfills its intended purpose and meets user requirements.” Verification refers to activities that ensure the product is built correctly by assessing whether it meets its specifications. Validation refers to activities that ensure the right product is built by determining whether it meets customer expectations and fulfills specific user defined intended purposes.
Software engineers have become competent at verification: we can build portions of systems to their applicable specifications with relative success.
However, we still build systems that don’t meet customers’ expectations and requirements. This is because people mistakenly combine V&V into one element, treating validation as the user’s operational evaluation of the system, resulting in the discovery of requirement errors late in the development process, when it’s costly, if not impossible, to x those errors and produce the right product.
Figure 1 illustrates how the validation process should be proactive and continuous—enacted prior to development and verification, with closure at the end of each phase. Validation is required whenever a requirements derivation process (that is, a translation of requirements from one domain to another) occurs.
... to read more articles, visit http://sqa.fyicenter.com/art/