Scenario Testing: Four Tips on How to Manage Effectively

By: Robin Gustafsson

Testing software has always been complex. The minute you add more than a handful of features to any system, theoretical complexity sky rockets.

All the buttons to click, links to follow, client browser versions, client bandwidth and what have you, will soon add up to a near infinite number of things you'd need to test. At the same time, actual users will only engage with a small fraction of those features.

But how does one bring some order to this complexity when, barring a few exceptions, 100% test coverage is basically impossible to achieve?

One approach is to turn to scenario testing. This is where you use a real or hypothetical story that describes how a user actually uses the application. It may sounds very similar to a test case, but a test case is typically single step, whereas scenario tests cover a number of interconnected steps.

A good scenario is one that is based on a credible story of a user performing a complex task. The scenario should be:

Critical to all stakeholders (i.e sales, marketing, management, customer support)
It should be obvious that the scenario must work as expected
The scenario must be easy to evaluate

Scenario testing is primarily thought of as a tool to facilitate feature testing, but performance is sometimes part of that.

Since application performance is very often a non-functional requirement, satisfactory performance is often assumed and lack there of is considered a bug - even if it was never mentioned in the requirements.

Therefore, scenario testing can be used to uncover important performance bottlenecks as well as test features.

Consider this test case for an e-commerce site.
Test case #1: Add valid coupon worth X% Steps:
Add one or more products to the cart
Go to the checkout page
Add coupon code 'TEST123' and click 'Add coupon'

Expected result:
Page refreshes. Message "coupon successfully applied" is visible
Discount clearly indicated in the cart summary section
Cart total is reduced by X%
Now, imagine that the test case is performed and confirmed to work during development. One of the testers makes a note saying that sometimes, the page refresh takes 4-5 seconds when you have more than 10 products in the cart, but it's not considered a major issue since it's affecting a very small number of users.

Now, consider an actual user, Lisa, as she uses the e-commerce site: Lisa gets a 15% coupon code for an eCommerce site she's used before and really likes. She decides to order a few things she needs and also asks her mother and sister if they need anything. While she's shopping, she talks once with her mother and three times with her sister to double check she get's correct amount, sizes and colors of all items.

Full article...

Other Resource

... to read more articles, visit http://sqa.fyicenter.com/art/