Interview Questions

Software Test Automation Tool Evaluation Criteria (1)

List of Software Test Tools

(Continued from previous question...)

Software Test Automation Tool Evaluation Criteria (1)

Ease of Use

  • Learning curve
  • Easy to maintain the tool
  • Easy to install--tool may not be used if difficult to install

Tool Customization

  • Can the tool be customized (can fields in tool be added or deleted)?
  • Does the tool support the required test procedure naming convention?

Platform Support

  • Can it be moved and run on several platforms at once, across a network (that is, cross-Windows support, Win95, and WinNT)?

Multiuser Access

  • What database does the tool use? Does it allow for scalability?
  • Network-based test repository--necessary when multiple access to repository is required

Defect Tracking

  • Does the tool come with an integrated defect-tracking feature?

Tool Functionality

  • Test scripting language--does the tool use a flexible, yet robust scripting language? What is the complexity of the scripting language: Is it 4 GL? Does it allow for modular script development?
  • Complexity of scripting language
  • Scripting language allows for variable declaration and use; allows passing of parameters between functions
  • Does the tool use a test script compiler or an interpreter?
  • Interactive test debugging--does the scripting language allow the user to view variable values, step through the code, integrate test procedures, or jump to other external procedures?
  • Does the tool allow recording at the widget level (object recognition level)?
  • Does the tool allow for interfacing with external .dll and .exe files?
  • Published APIs--language interface capabilities
  • ODBC support--does the tool support any ODBC-compliant database?
  • Is the tool intrusive (that is, does source code need to be expanded by inserting additional statements)?
  • Communication protocols--can the tool be adapted to various communication protocols (such as TCP/IP, IPX)?
  • Custom control support--does the tool allow you to map to additional custom controls, so the tool is still compatible and usable?
  • Ability to kick off scripts at a specified time; scripts can run unattended
  • Allows for adding timers
  • Allows for adding comments during recording
  • Compatible with the GUI programming language and entire hardware and software development environment used for application under test (i.e., VB, Powerbuilder)
  • Can query or update test data during playback (that is, allows the use of SQL statements)
  • Supports the creation of a library of reusable function
  • Allows for wrappers (shells) where multiple procedures can be linked together and are called from one procedure
  • Test results analysis--does the tool allow you to easily see whether the tests have passed or failed (that is, automatic creation of test results log)?
  • Test execution on script playback--can the tool handle error recovery and unexpected active windows, log the discrepancy, and continue playback (automatic recovery from errors)?
  • Allows for synchronization between client and server
  • Allows for automatic test procedure generation
  • Allows for automatic data generation

Reporting Capability

  • Ability to provide graphical results (charts and graphs)
  • Ability to provide reports
  • What report writer does the tool use?
  • Can predefined reports be modified and/or can new reports be created?

(Continued on next question...)

Other Interview Questions