In 2008, there were less than a thousand apps available through the iTunes store. Today, that number tops 1.2 million. There are more than a billion smartphones currently in use worldwide, with new devices being launched every single week. However, for those tasked with bringing a product to market, automated mobile testing isn’t widely adopted at this time.
The major drawback to automated mobile testing is the inability to replicate and repeat a test across multiple devices and operating systems. Although there are automated tests which will put a device or application through its paces, it becomes laborious and time intensive (and hence, more expensive) to set up that test to run on a majority of the options currently in use.
When you consider the challenge on a global scale, factoring in the multitude of technical specifications and security certifications that change from the United States to Europe to Asia and South America – as well as the need for distributed global teams skilled in each region’s unique requirements – the task borders on cost prohibitive for many middle-size and smaller developers. For those that have the big budget to partner with a testing partner capable of overcoming these challenges, they’re quickly reminded of certain limitations that exist within machine intelligence as automation removes human brainpower from the equation.
Here are three questions to ask when selecting a mobile testing partner to help you achieve a high level of automation without sacrificing the “boots on the ground” needed to ensure a successful launch.
1) Is your automation framework pre-built and customizable? Building a testing framework from scratch is complicated and requires up-front start-up costs. However, reliable options exist to leverage pre-built and reusable test automation framework.
At Lionbridge, for example, our existing VeriTestAF allows the ability to script once and run across different devices. It supports unattended execution both by scheduler and event triggering, such as the completion of a build. VeriTestAF is also both keyword and data driven so changes can be accommodated in a single location as opposed to revisiting every script that touches a function. A subset of a test can be run as well. This is helpful to review back-end functionality while a user-interface continues to evolve.
2) Do you have a global support team? Developers are facing increased pressure to launch their products across multiple regions within a short window of time. This requires a physical presence so selecting a partner without a strategically distributed network of secure, support testing centers will stall global growth.
In addition to US facilities, these regional hubs should be located in key cities, such as Dublin, Ireland; Warsaw, Poland; Mumbai, India; and Beijing, among other international hot spots.
3) How do you add human brainpower back in to the process? The current industry mindset is to consider testing projects in two parts – the work which takes place in the test center, which is then followed by in-country testing. This extends the process, slows time to market, and makes fixing bugs more expensive. But the two don’t have to be mutually exclusive. Seek out a partner capable of delivering a true end-to-end experience.
There are also highly effective, innovative approaches being used to complete in-country testing more affordably. One is the Business Process Crowdsourcing approach deployed by Lionbridge, where our on-demand network of experienced global crowd testers can be activated for specific projects. This on-demand nature, coupled with minimal infrastructure and overhead, allows the process to be completed quicker and usually at a lower cost than other approaches.
Every testing project is unique, and there is no one-size-fits-all approach. However, there are highly customizable automation frameworks, such as VeriTestAF, which can be quickly adapted to suit most mobile testing projects. This approach will help control costs and accelerate project ROI. The key, however, is asking the right questions to find the best partner.
Has your experience with automated testing been a success or a bust? Does your organization prefer an end-to-end approach or do you rely on multiple partners to handle each aspect of the testing phase?