With mobile application capabilities and usage
increasing, the application functions that are performed on a mobile device are
also becoming more complex in functionality and more critical to businesses.
The mobile mode of service delivery -- being closest to end users and always
available -- requires that developers pay particular attention to mobile
application performance testing. Even users expect more responsive performance
from mobile applications than they do from desktop apps.
So, it has become extremely important to
determine the performance that a mobile application's end-user will experience
before actually deploying the application onto a mobile platform. To do so,
mobile application developers need to performance test their applications with
an end-to-end application delivery model that tests for all the varying
scenarios on each component involved.
In this article, we propose such an end-to-end
approach to mobile application performance testing. Our aim is to present a
performance testing exercise that will help mobile application developers
accurately gauge performance from the end-user's point of view.
Mobile Application Testing by Type
It is important first to understand the types of
mobile applications developed today. Each type calls for a tailored performance
testing approach targeting the components and processes involved.
Testing Browser-Based Mobile Apps
Browser-based mobile applications might perform
differently as compared with the same application accessed via a desktop
browser. The causes of these differences include the bandwidth limitations of
the user's cellular network data plan and differing device configurations. The
advantage of a mobile browser-based application is the lower development cost,
as the mobile app developer needs to consider only how to make it compatible
with most mobile browsers.
While testing for performance, it is important to
replicate the user server load from mobile browsers too, in case additional
dedicated components are being deployed to cater to such requests. It is also
important to test Web page rendering on the target device as device
configuration (e.g. CPU, memory or browser rendering engine) can affect
end-user performance.
Testing Native Mobile Apps
Native mobile applications are installed directly
onto mobile devices, allowing users to access the applications quickly. The
disadvantage of native mobile apps is the development cost. Because the
applications are platform dependent, an app developed for one mobile OS won't
work for another. The developer is left having to develop and test the
application on all the platforms, such as iOS, Android, Blackberry, and so on.
This calls for thorough performance testing on each target platform as well as
on a set of devices in each selected platform.
Testing Hybrid Mobile Apps
Hybrid mobile applications are installed on mobile
devices just like native mobile apps but users can access them via either an
installed app or the device browser. The application is developed using Web
technologies and then wrapped in a platform-specific shell that allows the app
to be installed just like a native app.
As with native applications, the disadvantage of
hybrid mobile applications is development cost. To allow a user to access the
application via an installed app, the developer has to develop the application
for all the platforms mentioned in the previous section. Again, a performance
test strategy has to target the load generated by users of such hybrid apps as
well as gauge the on-device application performance.
Mobile Application Performance Drivers
An ideal mobile application performance test
should target every performance driver. Figure 1 presents the key components
governing a mobile application's performance.
Figure 1. Key Components Governing Mobile App Performance
For a typical application delivered to a mobile
device, the performance as perceived by the end user depends on two key
factors:
- Content delivery to device: This process involves the request delivery to the server, response generation at the server, and the response delivery back to the source device. Typical parameters that affect this are load on the server components, type of network involved, load on the network, and the external component behavior. These parameters can be handled and tested for by emulating the expected real world conditions on the server and network.
- Content rendering at device: This process involves displaying the response from the server at the source device. Typical parameters that affect this are type of platform, type and configuration of device, and the single user behavior patterns. These parameters can be handled and tested for by in-parallel, on-device application access and monitoring.