Comparing SilkTest and WinRunner - (3)
List of Software Test Tools
(Continued from previous question...)
Comparing SilkTest and WinRunner - (3)
Custom Objects
Note: The description of this feature, more than any other in this report, is limited in its scope and
coverage. An entire white paper could be dedicated to exploring and describing how each of
these tools deal with custom objects. Therefore dedicate several days to evaluating how each
of these tools accommodate custom objects in your specific applications.
To deal with a custom object [i.e. an object that does not map to standard class] both tools
support the use of class mapping [i.e. mapping a custom class to a standard class with like
functionality], along with a variety of X:Y pixel coordinate clicking techniques [some screen
absolute, some object relative] to deal with bitmap objects, as well as the ability to use external
DLL functions to assist in object identification and verification. Beyond these shared capabilities
each tool has the following unique custom object capabilities:
• SilkTest has a feature to overlay a logical grid of X rows by Y columns on a graphic that has
evenly spaced “hot spots”[this grid definition is then used to define logical GUI declarations
for each hot spot]. These X:Y row/column coordinates are resolution independent [i.e. the
logical reference says “give me 4th column thing in the 2nd row”, where that grid expands or
contracts depending on screen resolution].
• WinRunner has a built-in text recognition engine which works with most standard fonts.
This capability can often be used to extract visible text from custom objects, position the
cursor over a custom object, etc. This engine can also be taught non-standard font types
which is does understand out of the box.
Both tools offer support for testing non-graphical controls through the advanced use of custom
DLLs [developed by the user], or the Extension Kit [SilkTest, which may have to be purchased at
an addition cost] and the Mercury API Function Library [WinRunner].
SilkTest and WinRunner Feature Descriptions Version 1.00 (7/6/00)
2.9 Internationalization (Language Localization)
• SilkTest supports the single-byte IBM extended ASCII character set, and its Technical
Support has also indicated “that Segue has no commitment for unicode”. The user guide
chapter titled “Supporting Internationalized Applications” shows a straightforward technique
for supporting X number of [single-byte IBM extended ASCII character set] languages in a
single test frame of GUI declarations.
• WinRunner provides no documentation on how to use the product to test language localized
applications. Technical Support has indicated that (1) “WinRunner supports multi-byte
character sets for language localized testing…”, (2) “there is currently no commitment for
the unicode character set…”, and (3) “it is possible to convert a US English GUI Map to
another language using a [user developed] phrase dictionary and various gui_* built-in
functions”.
Database Interfaces
Both tools provide a variety of built-in functions to perform Structure Query Language (SQL)
queries to control, alter, and extract information from any database which supports the Open
Database Connectivity (ODBC) interface.
Database Verification
Both tools provide a variety of built-in functions to make SQL queries to extract information
from an ODBC compliant database and save it to a variable [or if you wish, an external file].
Verification at this level is done with hand coding.
WinRunner also provides a visual recorder to create a Database Checkpoint used to validate the
selected contents of an ODBC compliant database within a testcase. This recorder creates side
files similar to GUI Checkpoints and has a built-in interface to (1) the Microsoft Query
application [which can be installed as part of Microsoft Office], and (2) to the Data Junction
application [which may have to be purchased at an addition cost], to assist in constructing and
saving complex database queries.
Data Driven Testing
Both tools support the notion of data-driven tests, but implement this feature differently:
• SilkTest’s implementation is built around an array of user defined records. A record is a
data structure defining an arbitrary number of data items which are populated with values
when the array is initialized [statically or at runtime]. Non-programmers can think of an
array of records as a memory resident spreadsheet of X number of rows which contain Y
number columns where each row/column intersection contains a data item.
The test code, as well as the array itself, must be hand coded. It is also possible to populate
the array each time the test is run by extracting the array’s data values from an ODBC
compliant database, using a series of built-in SQL function calls. The test then iterates
through the array such that each iteration of the test uses the data items from the next record
in the array to drive the test or provide expected data values.
• WinRunner’s implementation is built around an Excel compatible spreadsheet file of X
number of rows which contain Y number of columns where each row/column intersection
contains a data item. This spreadsheet is referred to as a Data Table.
The test code, as well as the Data Table itself, can be created with hand coding or the use of
the DataDriver visual recorder. It is also possible to populate a Data Table file each time the
test is run by extracting the table’s data values from an ODBC compliant database using a
WinRunner wizard interfaced to the Microsoft Query application.. The test then iterates
through the Data Table such that each iteration of the test uses the data items from the next
row in the table to drive the test or provide expected data values.
Both tools also support the capability to pass data values into a testcase for a more modest
approach to data driving a test.
Restoring an Application’s Initial State
• SilkTest provides a built-in recovery system which restores the application to a stable state,
referred to as the basestate, when the test or application under test fails ungracefully. The
default basestate is defined to be: (1) the application under test is running; (2) the application
is not minimized; and (3) only the application’s main window is open and active. There are
many built-in functions and features which allow the test engineer to modify, extend, and
customize the recovery system to meet the needs of each application under test.
• WinRunner does not provide a built-in recovery system. You need to code routines to return
the application under test to its basestate—and dismiss all orphaned dialogs—when a test
fails ungracefully.
Scripting Language
Both tools provide proprietary, interpreted, scripting languages. Each language provide the usual
flow control constructs, arithmetic and logical operators, and a variety of built-in library
functions to perform such activities as string manipulation, [limited] regular expression support,
standard input and output, etc. But that is where the similarity ends:
• SilkTest provides a strongly typed, object-oriented programming language called 4Test.
Variables and constants may be one of 19 built-in data types, along with a user defined
record data type. 4Test supports single- and multi-dimensional dynamic arrays and lists,
which can be initialized statically or dynamically. Exception handling is built into the
language [via the do… except statement].
• WinRunner provides a non-typed, C-like procedural programming language called TSL.
Variables and constants are either numbers or strings [conversion between these two types
occurs dynamically, depending on usage]. There is no user defined data type such as a record
or a data structure. TSL supports sparsely populated associative single- and [pseudo] multidimension
arrays, which can be initialized statically or dynamically—element access is
always done using string references—foobar[“1”] and foobar[1] access the same
element [as the second access is implicitly converted to an associative string index
reference]. Exception handling is not built into the language.
The only way to truly understand and appreciate the differences between these two programming
environments is to use and experiment with both of them.
Exception Handling
Both tools provide the ability to handle unexpected events and errors, referred to as exceptions,
but implement this feature differently:
• SilkTest’s exception handling is built into the 4Test language—using the do… except
construct you can handle the exception locally, instead of [implicitly] using SilkTest’s
default built-in exception handler [which terminates the currently running test and logs an
error]. If an exception is raised within the do block of the statement control is then
immediately passed to the except block of code. A variety of built-in functions
[LogError(), LogWarning, ExceptNum(), ExceptLog(), etc.] and 4Test statements
[raise, reraise, etc.] aid in the processing of trapped exceptions within the except block
of code.
• WinRunner’s exception handling is built around (1) defining an exception based on the type
of exception (Popup, TSL, or object) and relevant characteristics about the exeception (most
often its error number); (2) writing an exception hander, and (3) enabling and disabling that
exception handler at appropriate point(s) in the code. These tasks can be achieved by hand
coding or through the use of the Exception Handling visual recorder.
(Continued on next question...)
Other Interview Questions
|