Software QA FYI - SQAFYI


By: Daniel J. Mosley

As client-server applications continue to replace mainframe-based systems, developers are looking for ways to adapt testing procedures for graphical user interfaces. Some advocate starting from scratch with new approaches; others, like Dan Mosley, see ways to amend old techniques and supplement them with new ones. Here, Mosley, who is the author of The Handbook Of MIS Application Software Testing: Methods, Techniques, and Tools for Assuring Quality Through Testing. Prentice-Hall, 1993, offers some tips from his upcoming book Testing Client-Server Application Software (Prentice-Hall, to be published fall 1995) and gives advice on selecting GUI test tools.

IN TRADITIONAL TESTING FOR mainframe-based information systems, which generally have a character-based user interface, a team of Information Systems development professionals test and implement the final design concurrently using a top-down technique. However, when it comes to testing client-server information systems, which typically have graphical user interfaces (GUIs), this approach as a whole is not sufficient. Some experts go so far as to suggest that you must create all-new testing methods for GUIs, but I see GUI testing strategy as a blend of amended traditional approaches and all-new ones.

Unlike mainframe systems, client-server systems are a form of distributed processing - data and processes are dispersed and replicated across miscellaneous platforms. This networked, cross-platform nature requires that you have much tighter process control during testing and that you pay a lot more attention to traditional forms of software testing such as Configuration Testing and Compatibility Testing.

Configuration Testing forces you to test the system in all the hardware and software environments in which you expect it to operate. Compatibility Testing ensures that there is a functionally consistent interface across those platforms. The interface may appear in any physical form as long as the same basic user behaviors elicit the same interface responses.

Furthermore, GUI testing possesses a major inherent deficiency that all forms of software testing have in common: It identifies and removes defects after the system is built, instead of preventing defects before it is built. For this reason, you should supplement GUI testing with established review techniques, such as Walkthroughs and Inspections, during the software analysis and design processes.

One system development method which is gaining wide popular appeal is Rapid Prototyping. When used during analysis, design, and construction of an information system, prototyping provides a "tangible" result very early in the development process. One aspect of client-server systems which lends itself well to prototyping is the GUI. This is so because Client-server development platforms, such as Powersoft's Powerbuilder and Microsoft's Visual Basic, have given us the ability to design and build the GUI early in the development of the system. This makes testing the GUI (including formal reviews such as Walkthroughs and Inspections) an activity that can be completed early too. The list in the box below gives an example of a structured GUI prototype test plan.

It is important to note that the test plan combines traditional testing approaches with the newer structure capture/replay approach. As items 1 -4 Illustrate, it is most important to first organize the GUI testing process. Specific (measurable) testing objects must be established and documented, specific characteristics of the modules being tested must be described, testing schedules and milestones must be set, and the member of the test team must be appointed and notified. All of these traditional testing activities can be supported by a testing tool which has a strong testing management component (they can be accomplished with paper and pencil methods if necessary). Items 5, 6, and 7 involve the design, construction, and execution of GUI test cases based on business scenarios (e.g. a typical day's work for a system user).These structured capture/replay activities must be completed using a testing tool which can organized, record, edit, and execute test cases, and provide a format for reporting test results. Finally, item 8 suggests that an important part of using a test tool is to evaluate how well the tool performed assessing where it could have worked better and how testers might have been better prepared to use the tool.

Obviously, my focus here is on tools - how their underlying techniques have changed, and how to select them. As the box shows, tools should be part of any GUI test strategy, but you should not ignore the importance of having an overall testing process. You can use this test plan to manage and control the testing with any commercially available GUI testing tool.

NEW TECHNIQUES. Several aspects of GUI testing are unique. Where GUI testing differs is that treating the interface as object-oriented makes good testing sense because the screen widgets (in the Windows environment these are known as "controls") and their supporting processes can be organized into classes of GUI objects which share the same set of properties. Any object in the same class, or any object from a sub-class of the class, should behave in identical ways. All replicated instances of the same GUI object should respond in exactly the same way to the same set of test data which was created to test the GUI objects of that class.

Another difference is the way input and output are recorded and saved. To test character-based interfaces, testers typically use a capture/playback tool. The tool records input as keystrokes and output as screen images, which are then saved to compare with I/O images in subsequent tests. This type of recording process is not suitable for GUI testing because it is at too low a level to handle the complexity of these interfaces. Unlike character-based interfaces, GUIs produce many kinds of graphical objects, which greatly increases the number of logic paths. Testing is further complicated because these objects can be present or absent, exist for any length of time, and appear anywhere on the desktop. As Kevin Farley notes " Windows programming, nothing changes as frequently as the user interface." A tool must consider which interface object has the input focus, the object's context relative to other state information, and the its position on the desktop (see "Software Testing for Windows Developers. Data Based Advisor, Nov. 1993, pp. 45-46,50-52.)." Thus, a capture/playback tool, which records at the level of specific screen events, has an impossible job.

Fortunately, a functional variation of the capture/playback paradigm, structured capture/playback has evolved specifically for GUI testing. Structured capture/playback is based on an internal (logical) view of the external interface activities. The application program's interactions with the GUI are recorded as internal events that affect specific screen objects, which the tool saves as scripts written in the vendor's proprietary language or possibly in Microsoft's Visual Basic or one of the C variants.

Many companies have begun producing structured capture/playback tools. Rather than list them here, and risk omitting important ones, I offer the following sources:

Linda Hayes, "Automated Testing for Everyone," OS/2 Professional, Nov. 1993, p. 51.

Stephen Quinn, John Ware. and John Spragens, "Tireless Testers: Automated Tools Can Help Iron Out the Kinks in Your Custom GUI Applications. InfoWorld,. Sept. 1993, pp. 78-79,82-83,85,

Dan Kara, "Ensuring Quality with Testing Tools is Good for Business," Application Development Trends, Feb. 1994, pp. 42-50.

Farley also gives tool reviews and listings of company names and addresses in the Data Based Advisor article.

Full article...

Other Resource

... to read more articles, visit