Mobile Testing

By:

Introduction
Most general software testing principals apply equally well to mobile solutions, although the number of tools available for mobile testing is much smaller, and there are a lot of extra potential problems your users can encounter that you have to test for.Many mobile solutions involve a significant hardware element in addition to the PDA, such as scanners, mobile telephony, GPS and position based devices, telemetry, etc. These extra hardware elements place additional demands on the tester, particularly in terms of isolating a bug to hardware or software.Mobile applications are often intended to be used by people with no technical or IT background, such as meter readers, milkmen, insurance sales people, on devices that have small screens, and either no keyboards or awkward keyboards. Good usability testing, carried out in conjunction with key users, in their own environment, is essential. I have seen a number of hand held projects fail for the reason that the end-user could not come to terms with the technology, even though the application was robust and met the functional spec. Many hand-held operating systems come in even more flavors than their desk top counterparts. I can think of seven flavors of Windows CE alone. Add to this that many enterprise PDA manufactures OEM the operating system, and update it regularly, you start to see the problems testing. Remember also that we don't have our faithful automation tools for regression testing here.

Let?s start with some Mobile Testing Basics.
Mobile Testing Basics
Mobile Device Testing is the process to assure the quality of mobile devices, like mobile phone, PDA etc. The testing will be conducted on both hardware and software.
And from the view of different procedures, the testing comprises R&D Testing, Factory Testing and Certificate Testing.
R&D Testing
R&D test is the main test phase for mobile device, and it happens during the developing phase of the mobile devices. It contains hardware testing, software testing, and mechanical testing.
Factory Testing
Factory Testing is a kind of sanity check on mobile devices. It's conducted automatically to verify that there are no defects brought by the manufacturing or assembling.
Certificate Testing
Certificate Testing is the check before a mobile device goes to market. Many institutes or governments require mobile devices to conform it's specifications and protocols to make sure the mobile device will not harm users' health and have the compatibility with devices from other manufactures. Once the mobile device passes the checking, a certificate will be issued to it.

Unique Challenges in Testing
Unlike the PC based environment, the mobile environment is constituted by a plethora of devices with diverse hardware and software configurations and communication intricacies.

This diversity in mobile computing environments presents unique challenges in application development, quality assurance and deployment, requiring unique testing strategies.

Mobile Buisness Applications can be classified into stand-alone applications and enterprise applications. On the other hand, Enterprise applications are built to perform resource intensive transactions that are typical of corporate computing environments. Enterprise applications also interface with external systems through Wireless Application Protocol (WAP) or Hyper Text Transfer Protocol (HTTP). The unique challenges in testing mobile applications arising from diversity of the device environment, hardware and networking considerations and Rapid Application Development (RAD) methodologies are explained below:

Diversity of the Device Environment
The realm of mobile computing is composed of various types of mobile devices and underlying software (hundreds of device types, over 40 mobile browsers). Some of the unique challenges involved in mobile testing as a result of this condition are:

Rendering of images and positioning of positioning of elements on screen may be unsuitable in some devices due to the difference in display sizes across mobile devices and models.

Exhaustive testing of user interfaces is necessary to ensure compatibility of the application.

Mobile devices have different application runtimes- For example, Binary Runtime Environment for Wireless (BREW), Java, embedded visual basic runtime are some of the runtimes commonly available in mobile devices. Applications should be tested exhaustively for the variations specific to runtime.

Hardware Configuration & Network related challenges

Mobile environment offers lesser memory and processing power for computing when compared with the traditional PC environment. Unlike the network landscape of PC environment, the network landscape of a mobile device may have gateways (access points between wireless internet and the cable internet). Some of the drawbacks of diverse hardware configurations and network landscape of mobile devices are:

Limitations in processing speed and memory size of mobile devices lead to variations in performance of applications across different types of devices.

Testing programs should ensure that the applications deliver optimum performance for all desired configurations of hardware.

Some devices communicate through WAP while some others use HTTP to communicate. Applications should be tested for their compatibility with WAP enabled as well as HTTP enabled devices.

The network latency (time taken for data transfer) will be unpredictable when applications communicate over network boundaries, leading to inconsistent data transfer speeds. Testing should measure the performance of applications for various network bandwidths.

Gateways in a wireless network may act as data optimzers that deliver content more suitable for specific devices. This data optimization process of gateways may result in decreased performance for heavy traffic. Testing should determine the network traffic at which the gateway capabilities will impact performance of the mobile application.

Rapid Application Development (RAD)
In order to deliver the benefits of faster time to market, RAD environments are used for mobile application development. Since the time taken for development is reduced by the introduction of RAD tools, builds will be available for testing much earlier. Therefore, RAD methodology imposes an indirect pressure on testing teams to reduce the testing cycle time without compromising on the quality and coverage.

Critical Success Factors
The critical factors that determine the the success of mobile testing program are:
Use of Test Automation

Use of emulators and actual devices

Testing for mobile environment and application complexity

Use of test automation

Testing of mobile applications is traditionally done by manual execution of test cases and visual verification of results. But it is an
effort intensive and time consuming process. Automating the appropriate areas of a testing program can yield quantifiable benefits.
Use of emulators and actual devices
Emulators can be beneficial for testing features of the application that are device independent. However, actual devices should be usedfor validating the results

Full article...

Other Resource

... to read more articles, visit http://sqa.fyicenter.com/art/