Software QA FYI - SQAFYI

Automated Testing of Win32 Applications on Multiple Operating Systems

By: AutomatedQA

Abstract This article tells about application testing on different versions of Windows operating systems via virtual machines working under Microsoft Virtual Server 2005, Automated Build Studio, which is AutomatedQA's build and release management system, and TestComplete, an automated testing tool by AutomatedQA.

In this article Testing on Different Operating Systems Testing on Virtual Machines Remote Control Over Resources Testing on Virtual Computers Sample Test Preliminary Operations Creating the Macro in Automated Build Studio Compiling the Application to be Tested Preparing Files to be Copied to the Virtual Machines Creating the Testing Submacro Submacro Parameters Submacro Structure Loading a Virtual Machine Testing Waiting for the End of Testing Processing Test Results Switching off a Virtual Machine Launching a Submacro Sending Notifications The Whole Macro The Results Summary Related Links Tech Papers

Write for Us!


Search Files

Testing on Different Operating Systems Any software developer may face a situation when his or her application works fine on one operating system, but closes abnormally or doesn’t work as expected on another operating system. Why does this happen? The reason is that the Windows operating system changes from version to version. The changes concern the application programming interface, security parameters, principles of working with memory, etc; moreover, an application may work incorrectly on localized versions of the operating system. That is why it is very important to test your application on different operating systems to get a reliable software product.

So, what should you do to start this kind of testing? First, you need to run several operating systems on your computer(s). Next, you need to create an application or script that will test your application. You can create a test script in TestComplete, but in this case, you will have to install the test engine on the computer that has the operating system for which you wish to test your application. Sometimes, it is inconvenient to launch a test manually, so you may need to automate testing. For this purpose, you need to have a testing system that will allow you to control processes on a remote machine.

It is often a good strategy to use several physical computers for this kind of testing. The computers must be within the same network environment. Here two questions arise: is it always convenient to use several computers? And is it always possible to have several computers at hand? It may happen that there are not enough computers on which you can perform testing. Even if your company has a lot of computers that can be used for testing purposes, they still may not be available to the tester, since they may have different locations.

One of unpleasant drawbacks that you might come across when testing on several physical computers is that if you test an application repeatedly, each installation may modify the operating system, so it becomes impossible to test a new version of the application correctly. For example, if the installation program works incorrectly, it may not remove some registry keys or files when uninstalling the application. If this happens, you will have to reinstall the operating system or abolish the changes manually, which is time-consuming and will slow down the testing process.

Testing on Virtual Machines A very good alternative to using several computers for testing is the use of virtual machines. A virtual machine is programmatic simulation of a computer’s hardware environment. It is possible to install the needed operating system on a hard drive of the virtual computer. You can install any operating system on a virtual machine - not just Windows only.

One of the most popular applications used to work with virtual machines is Microsoft Virtual Server 2005. It uses two files to store a virtual machine with one hard drive on it – a configuration file and a file containing an image of the hard drive. Storing an image of the hard drive in a file allows you to test your application on the operating system that was not changed by previous installations. For this purpose, it’s quite enough to copy both files of the virtual machine from some common repository in the network to a computer that has Virtual Server installed.

Virtual Server allows you to launch up to 256 virtual machines simultaneously. This doesn’t require a lot of powerful computers – you just need to have one high-end physical computer to work with Virtual Server and launch the needed number of virtual machines. A running virtual machine becomes available in the network automatically, so it’s possible to get access to its resources through the net. The rest of this article describes testing via virtual machines.

Remote Control Over Resources Virtual Server has no means to control processes running on supervised virtual machines. However, the fact that remote work with a virtual computer’s resources does not differ from the work with resources of a physical remote computer allows you to use standard means of remote control over processes on virtual computers. One of the most reliable tools, which is used on computers with Windows NT/2000/XP/2003 Server, is Windows Management Instrumentation (WMI). WMI is a Microsoft implementation of the WBEM protocol (Web-Based Enterprise Management) that defines the standards of access to data in enterprise networks. WMI allows you to safely control a number of resources of a remote computer: its processes, services, registry, etc.

For reasons of security, Microsoft did not implement the ability to remotely launch an interactive process via WMI. At first, this doesn’t appear to allow you to test the application’s user interface, but the RunInteractive.exe application shipped with Automated Build Studio makes it possible to launch interactive processes even if the application is running as a non-interactive process.

Testing on Virtual Computers AutomatedQA’s Automated Build Studio 1.4+ supports WMI and Microsoft Virtual Server 2005, which makes it possible to create macros that will perform automatic testing on several virtual machines.

Automated Build Studio's operations that support WMI allow you to control processes (application launching, closing, enumeration, status checking) and services (status checking and changing, enumeration), and to get information about registered COM classes on both local and remote computers. Contributing to Open Source Projects Contributing to an open source tool is one of the best ways to practice because it most accurately simulates a real project. By contribute, I mean that you can submit code, test code, write documentation and examples, or just be active on a mailing list and answer questions for people getting started. All of these activities will help some aspect of your testing because all of them expose you to different real problems on a software project.

Open source projects are good because you can contribute on your own time—or, if you're one of the lucky, while you're at work (a great example is the work that Atomic Object did with Systir). If you just can't seem to find the time or energy to contribute, you can also practice by downloading and using some open source tools. A good place to find tools to practice with is At the time of writing, they listed 207 tools.

Full article...

Other Resource

... to read more articles, visit

Automated Testing of Win32 Applications on Multiple Operating Systems