Testers: Time to gear up for mobile software testing

By: Colleen Frye

Testers: Time to gear up for mobile software testing

The economy has many businesses retrenching or in a holding pattern -- but mobile applications designed to be accessed via smartphones or personal digital assistants (PDAs) are poised to be one of the next big things, according to many experts. If so, what impact will that have on enterprise quality assurance (QA) and testing organizations?

"Good testing practices apply regardless of the platform," said author and testing consultant Judy McKay, but she and other testing pros say mobile apps will also pose some unique challenges.

For starters, "the mobile phone is a frontier-based mentality," said William Coleman, vice president of business development at LogiGear Corp., a software testing services company in San Mateo, Calif. "There are four or five operating systems all competing for supremacy," he said, and "very loose standards … that many phone manufacturers and app developers circumvent."

But despite the lack of standards -- and the down economy -- it appears mobile app development is forging ahead. In a study released in January, Evans Data Corp. found that 94% of corporate developers expect the development of wireless enterprise applications to either increase (47.6%) or stay the same (46.4%) this year, with the Asia-Pacific region leading the growth.

For the business user, "we're seeing enterprise apps being developed on enterprise standard OSes like WinMo [Windows Mobile] and RIM, but we don't see a rush to the others just yet. Enterprises trust Microsoft and RIM," said Coleman. According to the Evans Data study, 40% more developers plan to target Windows Mobile than Apple iPhone, and 46% more plan to target .NET (compact framework) than Google's Android platform.

Not only are there multiple operating systems to take into account, but testers will also have to deal with multiple versions of an operating system, various hardware devices and form factors, and the strength of a carrier's network connections and services.

"The amount of permutations does create a significant problem for testing and time to market," said Doron Reuveni, CEO and co-founder of uTest, a Software as a Service (SaaS) marketplace for software application testing based in Southborough, Mass. "The issues in mobile apps are quite significant compared to testing Web and desktop apps."

Andrew Reshefsky is a uTest tester who has worked on mobile email encryption programs for corporate clients. A big challenge, he said, is that users are on different networks, and "you have to figure out whether the issue is because of the network or the software."

Another issue is the device itself. "Every phone has an OS; some phones can update the OS, some can't. If the problem is with the phone you have to figure that out, because software developers don't like to fix problems that don't exist [with the software]," he said.

Testers also face the challenge of shorter development cycles. "The time to market is extremely short, so usability testing is a big thing," McKay said. According to Evans Data, 40% of wireless development projects take three to six months to complete, and 60% are completed in less than six months.

McKay added that there will be "more emphasis on performance, connectivity and upgradeability."

"When you have a captive user, they're stuck with the performance you give them, but with mobile devices they expect very fast performance, but you don't have a guaranteed level of connectivity," she said. If users are unhappy with performance, "there's a higher risk that the app will be completely rejected."

uTest's Reuveni said their customers are looking for two levels of testing. First, "they want some level of certification that this works on this variety of carriers, networks, phones, etc., with this version of the software, so there's a degree of comfort when they release it," he said.

Second, he said, is to look for flaws. "In mobile there's a lot of ad hoc exploratory testing, partly due to the apps being developed so quickly, but also the customer wants real user behavior, so they're looking for a combination of flaws and feedback from real users that understand and have seen a lot of mobile apps."

Test automation will be key
To meet the challenges of testing mobile apps, automation will be key, McKay said. "Any time you do performance load you've got to use tools," she said. LogiGear's Coleman added, "As complexity [increases], the requirements for testing get more demanding. The only way to capture that is through automation."

LogiGear specializes in test automation, and Coleman said they are working in the mobile area. "The big hole in this space is automation for mobile phones," he said.

It's a nascent area, said Manish Mathuria, founder and chief technology officer of InfoStretch Corp., an outsourcer of quality assurance, test automation, software development and mobile testing services. "One reason is the form factors of the different devices in use vary so widely; for an automation tool to cover all of them reliably is not an easy task," he said.

In terms of adopting automation, Mathuria said, QA teams have a long way to go as well: "Enterprise QA teams are sometimes barely equipped to manage automation well on the desktop side."

Mathuria said that today the great majority of mobile apps his company sees are consumer-facing, with games being the biggest area, followed by utilities, but everyone interviewed for this article agreed that the enterprise is likely to follow. "The smartphone is the new computer abstraction," Coleman said. "Ten years from now we won't be using laptops."

Are testers prepared for the mobile revolution? "Testers tend to be behind the development curve," McKay said. "Developers are out experimenting with new things, but testers may be deeply involved in the last project."

Her advice? "Keep an eye on what developers are doing and thinking, and the ops people. [Testers] can no longer pretend they don't need to know about performance of load and connectivity. There is a lot of expertise that will have to be developed, and this will be the time to get better."

Full article...

Other Resource

... to read more articles, visit http://sqa.fyicenter.com/art/