Especially when designing and debugging performance tests, test databases can become
dramatically overloaded with data. Periodically check to see if the data base is storing
unrealistic volumes of data for the situation you are trying to simulate.
Consider including invalid data in your performance tests. For example, include some users
who mistype their password on the first attempt but do it correctly on a second try.
First-time users usually spend significantly longer on each page or activity than experienced
The best possible test data is test data collected from a production database or log file.
Consider client-side caching. First-time users will be downloading every object on the site,
while frequent visitors are likely to have many static objects and/or cookies stored in their
local cache. When capturing the uniqueness of the user's behavior, consider whether that
user represents a first-time user or a user with an established client-side cache.
User abandonment refers to situations where customers exit the Web site before completing a
task, because of performance slowness. People have different rates of tolerance for performance,
depending on their psychological profile and the type of page they request. Failing to account for
user abandonment will cause loads that are highly unrealistic and improbable. Load tests should
simulate user abandonment as realistically as possible or they may cause types of load that will
never occur in real life -- and create bottlenecks that might never happen with real users. Load
tests should report the number of users that might abandon the Web site due to poor
In a typical Web site traffic pattern, when the load gets too heavy for the system/application to
handle, the site slows down, causing people to abandon it, thus decreasing the load until the
system speeds back up to an acceptable rate. Abandonment creates a self-policing mechanism
that recovers performance at previous levels (when the overload occurred), even at the cost of
losing some customers. Therefore, one reason to accurately account for user abandonment is to
see just how many users "some" is. Another reason is to determine the actual volume your
application can maintain before you start losing customers. Yet another reason to account for
user abandonment is to avoid simulating, and subsequently resolving, bottlenecks that
realistically might not even be possible.
If you do not account for abandonment at all, the load test may wait indefinitely to receive the
page or object it requested. When the test eventually receives that object, even if "eventually"
takes hours longer than a real user would wait, the test will move on to the next object as if
nothing were wrong. If the request for an object simply is not acknowledged, the test skips it and
makes a note in the test execution log with no regard as to whether that object was critical to the
user. Note that there are some cases where not accounting for abandonment is an accurate
representation of reality; for instance, a Web-based application that has been exclusively created
for an audience that has no choice but to wait because there are no alternative methods of
completing a required task.
The following are generally useful guidelines related to user abandonment: