|
Comparing SilkTest and WinRunner - (4)
List of Software Test Tools
(Continued from previous question...)
Comparing SilkTest and WinRunner - (4)
Test Results Analysis
• SilkTest’s test results file resolves around the test run. For example if you run 3 testcases
[via a test suite or SilkOrganizer] all of the information for that test run will be stored in a
single results file. There is a viewer to analyze the results of the last test run or X previous to
the last run. Errors captured in the results file contain a full stack trace to the failing line of
code, which can be brought up in the editor by double-clicking on any line in that stack trace.
• WinRunner’s test results file revolves around each testcase. For example if you run 3
testcases [by hand or via a batch test or TestDirector] 3 test results files are created, each in a
subdirectory under its associated testcase. There is a viewer to analyze the results of a test’s
last run or if results have not been deleted, a previous run. Double clicking on events in the
log often expands that entry’s information, sometimes bringing up specialized viewers [for
example when that event is some type of checkpoint or some type of failure].
Managing the Testing Process
• SilkTest has a built-in facility, SilkOrganizer, for creating a testplan and then linking the
testplan to testcases. SilkOrganizer can also be used to track the automation process and
control the execution of selected groups of testcases. One or more user defined attributes
[such as “Test Category”, “Author”, “Module”, etc.] are assigned to each testcase and then
later used in testplan queries to select a group of tests for execution. There is also a modest
capability to add manual test placeholders in the testplan, and then manually add pass/fail
status to the results of a full test run. SilkTest also supports a test suite, which is a file
containing calls to one or more test scripts or other test suites.
• WinRunner integrates with a separate program called TestDirector [at a substantial
additional cost], for visually creating a test project and then linking WinRunner testcases into
that project. TestDirector is a database repository based application that provides a variety of
tools to analyze and manipulate the various database tables and test results stored in the
repository for the project. A bug reporting and tracking tool is included with TestDirector as
well [and this bug tracking tool supports a browser based client].
Using a visual recorder, testcases are added to one or more test sets [such as “Test
Category”, “Author”, “Module”, etc.] for execution as a group. There is a robust capability
for authoring manual test cases [i.e. describing of each test step and its expected results],
interactively executing each manual test, and then saving the pass/fail status for each test
step in the repository. TestDirector also allows test sets to be scheduled for execution at a
time and date in the future, as well as executing tests remotely on other machines [this last
capability requires the Enterprise version of TestDirector].
TestDirector is also capable of interfacing with and executing LoadRunner test scripts as
well as other 3rd party test scripts [but this later capability requires custom programming via
TestDirector APIs]. Additionally TestDirector provides API’s to allow WinRunner as well as
other 3rd party test tools [and programming environments] to interface with a TestDirector
database.
External Files
When the tool’s source code files are managed with a code control tool such as PVCS or Visual
SourceSafe, it is useful to understand what external side files are created:
• SilkTest implicitly creates *.*o bytecode-like executables after interpreting the source code
contained in testcases and include files [but it unlikely that most people will want to source
code control these files]. No side files are created in the course of using its recorders.
SilkTest does though create an explicit *.bmp files for storing the expected and actual
captured bitmap images when performing a bitmap verification.
• WinRunner implicitly creates many side files using a variety of extensions [*.eve, *.hdr,
*.asc, *.ckl, *.chk, and a few others] in a variety of implicitly created subdirectories
[/db, /exp, /chklist, /resX] under the testcase in the course of using its visual recorders
as well as storing pass/fail results at runtime.
(Continued on next question...)
Other Interview Questions
| 