Interview Questions

Software Test Automation Tool Evaluation Criteria - (2)

List of Software Test Tools

(Continued from previous question...)

Software Test Automation Tool Evaluation Criteria - (2)

Performance and Stress Testing

  • Performance and stress testing tool is integrated with GUI testing tool
  • Supports stress, load, and performance testing Allows for simulation of users without requiring use of physical workstations
  • Ability to support configuration testing (that is, tests can be run on different hardware and software configurations)
  • Ability to submit a variable script from a data pool of library of scripts/data entries and logon IDs/password
  • Supports resource monitoring (memory, disk space, system resources)
  • Synchronization ability so that a script can access a record in database at the same time to determine locking, deadlock conditions, and concurrency control problems
  • Ability to detect when events have completed in a reliable fashion
  • Ability to provide client to server response times
  • Ability to provide graphical results
  • Ability to provide performance measurements of data loading

Version Control

  • Does the tool come with integrated version control capability?
  • Can the tool be integrated with other version control tools

Test Planning and Management

  • Test planning and management tool is integrated with

GUI testing tool

  • Test planning and management tool is integrated with requirements management tool
  • Test planning and management tool follows specific industry standard on testing process (such as SEI/CMM, ISO)
  • Supports test execution management
  • Allows for test planning--does the tool support planning, managing, and analyzing testing efforts? Can the tool reference test plans, matrices, and product specifications to create traceability?
  • Allows for measuring test progress
  • Allows for various reporting activities

Pricing

  • Is the price within the estimated price range?
  • What type of licensing is being used (floating, fixed)?
  • Is the price competitive?

Vendor Qualifications

  • Maturity of product
  • Market share of product
  • Vendor qualifications, such as financial stability and length of existence. What is the vendor's track record?
  • Are software patches provided, if deemed necessary?
  • Are upgrades provided on a regular basis?
  • Customer support
  • Training is available
  • Is a tool Help feature available? Is the tool well documented?
  • Availability and access to tool user groups

(Continued on next question...)

Other Interview Questions