<< End-user Response Times | Processor Time and Queue >>

Resource Utilizations

Considerations

Even though end-user response times are the most commonly reported performance-
testing metric, there are still important points to consider.
·

Eliminate outliers before reporting. Even one legitimate outlier can dramatically
skew your results.

Ensure that the statistics are clearly communicated. The difference between an
average and a 90

percentile, for example, can easily be the difference between "ship

it" and "fix it."

Report abandonment separately. If you are accounting for user abandonment, the
collected response times for abandoned pages may not represent the same activity as
non-abandoned pages. To be safe, report response times for non-abandoned pages
with an end-user response time graph and response times and abandonment
percentages by page on a separate graph or table.

Report every page or transaction separately. Even though some pages may appear
to represent an equivalence class, there could be differences that you are unaware of.

Resource Utilizations

Resource utilizations are the second most requested and reported metrics in performance
testing. Most frequently, resource utilization metrics are reported verbally or in a
narrative fashion. For example, "The CPU utilization of the application server never
exceeded 45 percent. The target is to stay below 70 percent." It is generally valuable to
report resource utilizations graphically when there is an issue to be communicated.

Exemplar for Stakeholders

Figure 16.3 Processor Time