Debugging Quality Control
By: Stephen Swoy
* As regulators clamp down, business users want a hands-on approach to software QA. Some in the industry are pushing for quality-level agreements.
* Developers say traditional metrics donít always work. Many remain in the dark about their own companiesí internal QA processes. A defined QA process, however, may make their jobs easier.
* A growing trend among Global 2000 companies is to hire quality-control managers and develop QA programs outside their IT depts.
Bugs and other defects are inevitable byproducts of software development. No one disputes this. Beyond that, the consensus usually breaks down.
The disconnect between developers and business users is almost a tradition. Business customers often give short shrift to the concerns of developers. They want software that can handle their business needs, even if those requirements evolve or change. That's why last-minute change requests and eleventh-hour feature or functionality additions are a fact of life for enterprise codejockeys. These down-to-the-wire disruptions may account for defects or bugs that ultimately get fingered by quality assurance testers.
A productive working relationship with line-of-business customers may get even harder to achieve. Sarbanes-Oxley and the current regulatory climate require higher levels of accountability. Not surprisingly, business users want a more hands-on approach to software QA.
As a result, a growing number of codejockeys are grappling with software QA initiatives. They're being asked to take on new responsibilities for testing and other QA-related activities. Why is this a problem?
Because when it comes to QA, many developers are in the dark. Many programmers can't tell you how their companies assess the quality of the software they produce--and in some cases, they don't care to know. Number of X and Y Theodore Woo, a production support engineer with a global IT services firm, is working for a telco that has outsourced most of its IT operations. Woo's firm won its outsourcing pitch by promising to reduce the telco's IT costs and improve the overall quality of its in-house software dev, in part by bringing its developers up to CMMI Level 3 speed.
As far as Agitar Software is concerned, the success of service enablement and a host of similar initiatives hinges on software quality. "If you are going to base your success on a service provided by someone else, how do you begin that relationship?" asks Jeffrey Fredrick, a principal with Agitar. "We think a key part of this relationship will be a transparency into the development process that is now almost unknown, and that quality-level agreements will be common."
Think of a QLA as a lot like a service-level agreement. It's a contract, more or less, that specifies requirements, expectations, responsibilities and other potential sticking points. QLAs can have teeth, too, because they may prescribe penalties or sanctions, monetary and otherwise, for quality breaches.
Agitar's Open Quality Initiative is highlighting a disturbing trend in commercial and enterprise software development. As Fredrick and other Agitar officials see it, software defects are now accepted as inevitable by developers and consumers alike. This in turn has precipitated a serious degradation in software quality.
To remedy this situation, Agitar is calling upon vendors to provide data about their internal software-quality efforts. By the same token, officials say customers should demand to see quantitative measurements of quality in the IT products and services they consume. Customers should ask for and receive QLAs from their software vendors just as they commonly do with SLAs.
It's a view that some in the industry are warming up to. "This is something we're starting to see a lot more," says Mark Eshelby, quality solutions product manager for testingtools specialist Compuware. QLAs provide a good way to make sure the project's start and finish criteria are reasonably well understood by both customers and programmers. The business user's participation in determining what level of quality and what criteria are going to be used to measure the quality of the finished project raises a potential issue because the concept is still evolving, Eshelby says.
A key QLA proving ground is the outsourcing arena. A surprising number of outsourced software dev projects don't deliver the goods, leaving orgs holding the bag. QLAs could give outsourcers more clout: If the finished project doesn't conform to the requirements set forth in the QLA, the outsourcing provider doesn't get paid.
It's no more unreasonable than some of the steps orgs currently take to protect themselves. However, it might not be as practicable, given the potential difficulty of enforcing contracts or judgments against offshore providers.
When Christopher Secord, a senior app dev with Wake Forest University, interviewed for a job with General Electric in 2001, he asked GE officials if they had notched contractual agreements with their outsourcing providers in India to protect the company in the event of disaster.
"Instead of an agreement that says, 'This...app has to meet the following metrics,' they broke everything up," remembers Secord, "as in 'User interface module A has to meet these requirements; user interface module B has to meet these other requirements.' The absolute worst-case scenario, they hoped, was that only one module would be a failure and they could replace it, with another outsourced firm if necessary."
Given the popularity of SLAs, adoption of QLAs might seem like a foregone conclusion. Not everyone is comfortable with that idea, however.
"I think the bottom line is that business people and developers have to have a relationship in which they can trust that the other is working toward the goal of getting a quality product out the door," says Ken Auer, a principal with custom software dev house Role Model Software. " A QLA seems like a bad way to define that relationship. It says, 'We, the business, have reasonable expectations and you, the developers, are incompetent or negligent if you don't meet them perfectly.' The reality is that the business people are taking an educated guess at what they would like to have done, and the developers are taking an educated guess as to the best way to get it done. They are both very prone to error. If they accept the reality, they can work together to reduce it."
... to read more articles, visit http://sqa.fyicenter.com/art/