Software QA FYI - SQAFYI

Using Intelligent Test Automation Techniques


If you have been on a test automation project, it's very likely that you have heard one or more of these comments. 'What??? Only 45 % of the manual test cases have been automated.' I thought this would be compatible with all the browsers?' Why can't I run this on different versions of the Windows OS?' 'I thought there would not be any need for manual testing after we completed the automation.' You mean we will have to upgrade the scripts each time the product changes. I thought it was a one-time job.'

These statements illustrate several problems that plague test automation projects:
* A single tester performing the dual role of manual and automation. This keeps it from getting the time and focus it needs.
* Unrealistic expectations from test automation. The most common one being, test automation a simple activity that required only record and playback.
* A lack of experience in the testing team in which things are made worse by a high turnover.
* An unhealthy shift of focus from testing the product to automating the testing just for the fun of it. Many find automating the testing more interesting than testing it. The outcome in such cases rarely contributes to the test effort.

In this article we will try to address issues faced by the test automation team and attempt to present a few feasible solutions. We will look at why test automation projects get shelved regularly and how the Generic Approach and the 'Neural Networks' approach could be the answer to your test automation woes.

Why do test automation projects get shelved?

There are primarily two reasons for test automation projects to get shelved. The first and the foremost are all the unrealistic promises dished out by the people managing test automation projects. People are often under a wrong impression that test automation is a piece of cake. Typical approach is to record and playback the test scripts, without giving any thoughts to the selection of tool(s), automation architecture and feasibility of the entire exercise. The second and equally important one is the inability of test automation to effectively replace manual testing.

Misconceptions surrounding test automation.

Some of the issues that we discussed in the previous article were'
* Test Automation tools are difficult to use.
* Test Automation is very easy. Just record the scripts at any given point of time and replay them whenever you want to.
* An unreasonably long time span is required to train the manual test team in usage of the tool(s).

While a more detailed explanation of these can be obtained at 'An Introduction to Software Test Automation', this list can be further extended'

* Automating manual test cases even before the unit testing is complete. Although this might seem strange and practically impossible, it is in fact quite common, thanks to an over zealous test automation team.
* Constructing test automation suites on the fly, without bothering to correlate the test automation scripts to the existing state of the product or the test cases.
* In case of situations where the automation scripts fail / terminate abnormally, they are often unable to start from the point where they left off. This causes the scripts to be rerun from scratch, resulting in loss of valuable testing time / effort.

Inability of test automation to effectively replace manual testing
In most of the organizations test automation is not considered as a feasible alternative to manual testing. Usually, it is a marketing gimmick used to entice customers or a prop supporting the organization's claims of being a company that works on cutting edge technology. In reality, with each passing day the test automation software is shelved and the focus shifts back to manual testing.

Reading this it is but natural for a few questions to popup in one's mind. Is test automation worth the investment? Can test automation ever replace manual testing?

Test Automation can be a feasible alternative to manual testing, provided that the architecture of the test automation suite is good enough and the scripts posses a certain amount of intelligence. Successful "Test Automation" is not rocket science; it just requires the right blend of planning, technical know-how and innovation.

Intelligent Test Automation
Test automation in its current form is not an alternative to manual testing. It is crude, ineffective and possesses no inbuilt intelligence what so ever. If test automation is to ever replace manual testing (even to a reasonable extent) it must be adaptive and intelligent. These are the two main qualities that separate a manual tester from a test automation script. Let's now look at two complementary approaches that attempt to bridge this gap between manual and automated testing.

Generic Test Automation
I'm sure that reading these words, the first question that will come in mind is: - Generic Scripts? Is this guy reinventing the wheel? No!!! I am not. Writing generic scripts does not only mean clubbing the common functionality together. It means writing scripts, which are essentially adaptive in nature. This is illustrated by the following example.

Consider a scenario in which you have been given the task to automate manual test cases used for testing a website. This website consists of half a dozen modules and you need to automate test cases for all of these. In the traditional approach using the functional decomposition method, the scripts are usually written in the following manner.

The above two figures clearly illustrate the differences between the two approaches to test automation. The traditional approach is analogous to an automated assembly line requiring manual intervention at a dozen places. This rather defeats the purpose of automation, ultimately resulting in even lesser productivity and efficiency. On the other hand, the generic approach derives all the benefits of test automaton resulting in less manual intervention and a marked increase in productivity and efficiency. Like any other methodology the generic approach has its own share of disadvantages. However, these are more of research areas rather than limitations.

Advantages of Generic Approach:
1. A single script can be used for majority of the test cases, barring a few exceptions.
2. Error handling is much more robust in these scripts, allowing unattended execution of the test scripts.
3. Turn around time for such scripts is extremely fast.
4. Only the test data and not the scripts need to be updated for future releases of the same product.
5. This is a one-time job in the true sense.

Disadvantages of Generic Approach:
1. Technical and highly skilled personnel are required to create and maintain the scripts.
2. Creation of the initial framework and libraries can be time consuming.
3. In case of complex products, embedding business logic in such scripts may prove to be a bit tedious. Such scripts are more suitable for sanity checks.
4. Though these scripts are generic and highly reusable, they are still dependent on manual input and they lack the intelligence of a manual testing team.

Thus, we can see that 'Generic Test Automation' not only reduces the scripting efforts, it also offers a chance to the test automation team to deviate from the traditional approach, resulting in a greater scope for innovation. Such scripts are truly adaptive in nature as they are constantly learning on the job. Such scripts are ideal for regression testing products across a large number of builds. The only thing these scripts lack in is 'intelligence'. Though these scripts are adaptive in nature and bring us a step closer in our goal to replace manual testing with automation testing, they are not intelligent enough to completely replace manual testing. The following section outlines a complementary approach in an attempt to bridge this gap further.

Test Automation using Neural Networks
Before we dwell on the usage of neural networks in test automation let us have a quick look at neural networks. Before we proceed please keep in mind that this article is about the usage of neural networks in test automation and not about neural networks. The sole reason for including this section in the article is to provide a brief introduction of neural networks.

Neural Networks
A Neural Network is a computer technique modeled on the supposed structure and operation of the brain and is involved in processing information, making rational decisions and initiating behavioral responses.

In other words, a neural network is a computer software / hardware that attempt to simulate a model of the neural cells in animals and humans with the sole purpose of acquiring the intelligence embedded in these cells. The biggest strength of a neural network is its ability to learn by example. Used in many commercial applications for pattern recognition, neural networks could be of tremendous use in test automation.Essentially, a neural network is nothing but a group of neurons connected together. Think of a neuron as a program or even better as a class, that accepts one or more inputs and produces one output.

A typical neuron has 2 modes of operation: the training mode and the using mode. In the training mode a neuron is taught how to output a value, based on the input value(s). In the using mode, a neuron scans a given input and its associated output equals the current output. If the neuron does not recognize the input as a part of its training program, it applies firing rules to determine the output. Firing rules are nothing but a set of instructions to help the neuron react sensibly to all the inputs, irrespective of whether they were a part of its training program. Some of the popular neural networks are Forward Connection Neural Network, Hopfield Network, Brain-State-in-a-Box, the Kohonen Network and the Back Propagation Network. The Back Propagation Network is a type of neural network in which a training sample is presented to a neural network.

In order to bridge the gap between manual and automation testing it is quite important for the automation scripts to possess a certain amount of intelligence. Naturally, this intelligence has to be fed into the script(s). The easiest and possibly the most efficient method would be to use the test automation scripts in conjunction with neural networks.

Apart from being an active area of research, today, neural networks are used in a variety of commercial applications including character recognition and image recognition, stock forecasting etc. Also, neural networks are a concept, therefore tool/technology independent. Hence, effective usage of neural networks is possible, regardless of the software test automation tool/technology being used. Secondly, neural network programs are typically modular in nature. Hence, the same set of functions and procedures can be plugged in across various scripts. The subsequent part of this section will provide an insight on the usage of neural networks in test automation.

Full article...

Other Resource

... to read more articles, visit

Using Intelligent Test Automation Techniques