In order to ensure a high quality response to market demand for mobile applications, they need to be tested on real mobile devices, and developers will also require test automation capabilities, a recent survey underlines.

Mobile Apps Insufficiently Tested on Real Mobile Devices

Information technology research and advisory firm Gartner has predicted that by 2017 mobile applications will be generating total revenues of $77 billion a year. However, the quality of mobile apps and websites will need to improve in order to meet the expectations of increasingly connected users, which means that the work of applications testers is becoming ever-more crucial. Michael Sowers, CIO and Quality Assurance (QA)/Test Senior Consultant at Florida-based Software Quality Engineering (SQE), underlines that “ensuring that these applications are adequately tested is a key challenge for Q/A/Test Teams worldwide.” Meanwhile Keynote, a Californian firm which provides cloud-based testing, monitoring and analytics for mobile and web, has recently carried out, in tandem with SQE, a survey among more than 1,600 mobile professionals, to obtain an overview of mobile app and website testing and the challenges the sector is facing.

Testers’ tough task

The Quality Assurance experts see the availability of mobile testing tools as the greatest challenge facing QA organisations today, this aspect scoring 6.6 on a scale of 1 to 10 (1 = not at all important; 10 = most important). ‘Having enough time to test’ and ‘access to mobile devices’ are the next two most serious challenges, the survey reveals. Consumers are always demanding “more functionality, faster, better and cheaper,” stresses Michael Sowers. In addition, 47% of the people testing mobile applications and websites say that functional testing is their biggest concern, followed by performance testing, which was reported as a major headache by 25% of the QA experts polled. To carry out successful functional testing, survey respondents ranked ‘easy access to many device models’ as the most important factor, giving it an average score of 7.91 on the 1 to 10 scale. Respondents rated ‘easy automation capabilities’ as the next most important feature, followed by ‘performance metrics’.

Key challenges for the testing process

“Today’s mobile users are difficult to satisfy and apps are easy to uninstall,” points out Rachel Obstler, Senior Director of Product Marketing at Keynote. The survey shows that expectations regarding the quality of apps for mobile platforms are higher than those for embedded (desktop) platforms. Therefore great care needs to be taken with the quality of mobile testing if the developers want to create loyalty among users. However, the survey found that testing time allocated to mobile releases is the same as for non-mobile releases. Rachael Obstler believes that “in order to meet mobile market demand for websites and applications, implementing processes that include real device testing and automation capabilities will be key to meeting rising expectations.” However, only “58% of respondents said that most of their mobile app or web testing is done on real devices,” and only “14% of respondents have automated more than 50% of their mobile testing,” states Keynote’s infographic. The survey also reveals that while mobile website and app testing teams at development organisations are generally ‘distributed’, decisions regarding test tool selection are most often made by a centralised quality assurance group, perhaps leading to less coherent decisions on these issues.

By Eliane HONG