Web Testing

By: Mitch Allen

How do you test a Web site anyway? That's the first thing I asked myself when I became the SQA Manager for a dot-com. I knew how to build Web sites, and I had tested several commercial products. It would be easy right? Nope. Not only is it hard, but it can also be expensive.

Performance and Scalability

If you are managing Web site testing for an application that will be used by a large number of people, your number one concern will be performance testing. When several people start using the Web site at the same time, performance may degrade, database transactions might overlap, etc. Your job is to make sure that that doesn't happen.

The first thing you should ask in an interview for Web testing is "will it scale?" Now what does that mean? You want to work on a Web site that is "n-tier." That means that it has at least three layers (tiers). The layer that the user of the site sees is the GUI (Graphical User Interface) layer (also referred to as the front-end or presentation layer). Behind that, in the middle is the middle tier. This is where all the business logic happens. It's also referred to as the application layer. This is where the application servers go. Behind that is the database tier also referred to as the back-end.

In an n-tier architecture, the GUI layer should be concerned with nothing but rendering the Web page and passing transaction requests to the middle tier. No heavy duty calculations should be performed in the user's browser. It should happen on the Web site in the application layer.

The GUI layer must never, ever, ever talk to the database directly. (Side bar: I'd like to point out that I violate these rules everyday - but I'm not developing large commercial n-tier sites).

The middle tier is the heart of the application and the key to scalability. When a user makes a transaction, like filling out a form on the GUI layer, the middle tier is where the transaction is handled. The middle tier will talk to the database and do some of the transaction processing. In J2EE architecture, this layer is written using Enterprise JavaBeans. In .NET systems, this layer may be written in a language like Visual C#.

The database (back-end) can also play an important role in transaction processing. The first step in improving a Web site's performance can be to move as much of the middle tier logic as possible to stored procedures in the database.

For a Web site to scale, it must be determined how many users a middle tier server can handle. This is done through performance testing. If the middle tier is designed properly, adding another server will double the amount of users that the system can handle. If you can serve up 1 transaction per second for 500 users using 1 middle tier server, then 3 servers should be able to maintain that rate for 1500 users (500 x 3). Though realistically there will always be an acceptable amount of degradation as you add a new server.

No Web site can handle an infinite amount of users. Performance requirements have to be realistic. You may have heard in the news about Denial of Service (DoS) attacks. This is basically performance testing run amok. Hackers hijack a bunch of servers to act as agents and flood a server with transactions until they overwhelm the system. Instead of breaking the system, your job is to determine how many users it can handle before it breaks.

Performance Testing

To test site scalability and performance, you need a testing product that can emulate x number of users hitting your site. Tools to do this range from being free, to $100,000 depending on how many "virtual users" you want to emulate. I could go on about how ridiculous it is to charge an extra $20,000 just to up an internal counter, but I'll refrain. Just be careful when shopping for tools.

Here is a list of some performance tools:

Segue Silk Performer
Mercury Interactive LoadRunner
Apache JMeter

Be forewarned that some of the more expensive tools have very restrictive licensing software. I was amazed to find out that after spending a fortune on one product that only one of my employees at a time could run the editor to write tests! They expected me to spend another $10,000 to get a special meter to allow multiple editors (I declined).

You will also need to invest in at least one machine that will act as an agent, emulating hits against your test servers. I've used as many as three. Your vendor can tell you what you will need.

Staging Servers

Before you launch into a testing effort, you should find out if there is enough money in the budget to emulate the production environment. You don't want to do tests against the live site.

You need to setup a "staging" environment which emulates your production environment on a smaller scale. You will need a router, just like the one in production (or if money is really tight, something as close to it as possible), at least two Web servers and two middle tier servers and one database server. And don't forget the firewalls. If the hardware doesn't match the production environment, then testing on the staging servers won't help you find problems in the production environment. You need to work closely with the production team to make sure that your test environment emulates every bit of hardware and software in the production environment.

Your staging servers need to exist with your performance testing environment in an isolated network. Usually a patch panel is setup so you can pull one plug and isolate a rack full of servers at will. When doing performance testing, all numbers will be useless if your environment is suffering from noise not only from the corporate net, but the Internet as well.
And lock them down! Make sure developers don't have ftp, write, or PC Anywhere access to your servers. Otherwise unseen changes to your staging environment could wreak havoc on your test results.

Routers, Firewalls and SSL

Large Web sites don't have just one Web server. When you visit the site, a router will redirect you to one of many servers. The job of the router is to balance the load by trying to evenly distribute users amongst the many servers. In order for the system to work properly, all Web servers need to be exact copies of one another. So that users can get to the GUI layer servers, they need to be outside the company firewall.

Even if you are just working on a Web application destined for internal use, users may still want to use it from outside the firewall. For example, your sales force may need access from a client site to give a demo or access the application. Regardless, you must test using a firewall. Firewalls go between the GUI layer and middle tier and it's not uncommon to have a firewall between the middle tier and the back-end. You will find that an application that worked fine in an open environment collapses once a firewall is introduced. Firewalls act as choke points which could cut off or interfere with the communication between tiers.

Secure Socket Layer (SSL) is another issue that you need to deal with. If you've ever entered your credit card number online and noticed a little lock symbol in your browser, that's SSL working. Another sign is when the Web address changes from "http" to "https." Private SSL servers require a complex procedure to install a special certificate for access. Users of a commercial site should not have to go through such a procedure to create a secure connection. You need to verify that external users can access the secure layer without a problem.

Cookies and Session Management

The Web is a stateless environment. This means that a lot of data isn't stored on the client side. When you visit a site and "talk" to a Web server, it doesn't remember who you are. Though you can help it remember if you let it set a "cookie" in your browser. This is how sites remember your name, etc. When you visit the site again it asks your browser for the cookie, which may contain something like your login ID.

Cookies are good for long term memory, but for short term memory, Web applications use session management. Session management is sort of like a cookie, but it will expire after a certain amount of inactivity (like 30 minutes). Think of it like a Session ID.

As you surf a site, you may end up talking to several servers at once. Each transaction may be dished out to a different server. As you move from server to server, your session information needs to stay with you or the next server will have no idea what you are talking about. For example, when you get to the checkout server, session information will help it remember what items you ordered. For testing, you need to make sure that as a user moves through the system that session information is carried with them and hasn't expired in a reasonable amount of time.

Components and Configuration Management

When trying to setup bug tracking you may come to a horrifying conclusion. The concept of a "build number" for the overall product may not work. Web site development can be very fluid. People will update script, image, and source files through FTP or file copying without even thinking about. The concept of "builds" may even be foreign to some Web developers. You need to work with development to determine what defines a build when entering bugs. Ideally your Web site will be broken up into components that can be tested in isolation and combined and rolled out as sets.

When testing, you want to be able to certify components and component sets. For example, you may certify that Contact Form 1.1 works well with Order App 1.3, but not 1.2. Because of the fluid nature of the Web, the site probably won't be updated all at once. Developers will usually want to "drop in" a component to the live site or a client's site. That's why configuration management and certification are so important. Before Component X 2.3 is dropped in with Component Y 2.4, you need to certify that everything will still work.

If your site uses ActiveX or some other technology to install components on the browser side, you need to deal with that as well. If your application uses browser side objects, find out from your development team if you can test them directly using a language like JavaScript.

Testing Under the Hood

The GUI layer often talks to the middle tier by tossing back and forth XML files. The trend is moving towards hiding the XML files as Web Service calls. You should ask your development team if the Web Services are published in the form of an internal or external document. If you have a programming background, you could emulate the GUI layers communication with the middle tier by automating Web Service calls. If you are comfortable with SQL, you could talk to development about unit testing stored procedures in the database layer.

Conclusion

Testing an enterprise level Web site can be an overwhelming and expensive task. Before embarking on such an effort, it's important to make sure that upper management will commit the time, money, and resources required. Given the proper budget, schedule, knowledge, and commitment from Development, you should be able to handle the task.

Mitch Allen has been working in SQA since 1989, specializing in test automation. For more information please visit his Web site at http://www.mitchallen.com.

Article © 2003 Mitch Allen