Factors to be considered during mobile testing

In many aspects, an approach to perform any type of testing is not so different from mobile automation testing. From methodology and experience, while working with the actual testing tools, what testers have learned in testing can be applied to mobile automation testing.

So, a question might come to your minds that then, where does the difference lie and how should you accommodate these differences? So, following this topic, we will see some of the factors that are highly relevant to mobile automation testing and require particular attention, but if handled correctly, then we can ensure a successful mobile testing effort.

Some of the factors that need to be taken care of in testing mobile applications are as follows:

• Testing for cross device and platform coverage: It is not feasible to test an application on each and every available device because of the plethora of devices that support the application across different platforms, which means you have to strategically choose only a limited, but sufficient set of physical devices. You need to remember that testing on one device, irrespective of whether it is of the same make, same operating system version, or uses the same platform cannot ensure that it would work on any other device. So, it is important that, at the very least, most of the critical features, if not all, are tested on a physical device. Otherwise, the application always runs a risk of potential failure on an untested device, especially when the target audience for the application is widespread, such as for a game or banking application.

  Use of emulated devices is one of the common ways to overcome the issues of testing on numerous physical devices. Although this approach is generally less expensive, we cannot rely completely on the emulated devices for the results they present, and with emulators, it may be quite possible that test conditions are not close enough to the real-life scenarios.

  So, an adequate coverage of different physical devices is required to test these following variations, providing sufficient coverage in order to negate the effects of fragmentation and have sufficient representation of these various factors:

  • Varying screen sizes
  • Different form factors
  • Different pixel densities and resolutions
  • Different input methods, such as QWERTY, touch screen, and more
  • Different user input methods, such as swipes, gestures, scrolling, and many more
• Testing different versions of an operating system of the same platform: For thorough testing, we need to test the application on all major platforms, such as Android, iOS, Windows, and others, for the target customer base, but each one of them has numerous versions available that keep on growing regularly. Most commonly, testing automation on the latest version of any operating system can be sufficient, as the operating systems are generally backward compatible. However, due to fragmentation of the Android OS, the application would still need to be tested on at least the most commonly used versions besides the latest ones, which in some cases may be significantly behind the latest version. This is because there may be many Android devices that are on an earlier version of Android and are not supported by the latest versions of Android.
• Testing of various network types and network providers: Most of the mobile applications, such as banking- or information-search-related applications require network connectivity, such as CDMA or GSM, at least partially, if not completely. If the application talks to a server about the flow of information to and fro, testing on various (at least all major) network providers is important. The network infrastructure used by network providers may affect data communication between application and the backend. Apart from the different network providers, an application needs to be tested on other modes of network communication, such as Wi-Fi network as well.
• Testing for mobile-environment-specific constraints: The mobile environment is very dynamic and has constraints, such as limited computing resources, available memory, in-between calls or messages, network switching, battery life, and a lot of other sensors and features, such as accelerometer, gyroscope, GPS, memory cards, camera, and others, present in the device, as an application's behavior depends on these factors. An application should integrate or interact (if required) with these features gracefully, and sufficient testing needs to be carried out in various situations to ensure this. However, oftentimes, it is not practically feasible to recreate all permutations and combinations of these factors, and hence a strategic approach needs to be taken to ensure sufficient coverage.
• Testing for the unpredictability of a mobile user: A tester has to be more cautious and should expand the horizon while testing the applications. They should make sure that an application provides an overall good response to all users and a good user experience; hence, User Experience (UX) testing invariably needs to be performed to a certain degree for all mobile applications. Since, a mobile application's audience comprises of various people ranging from nontech people to skilled technical users and from children to middle-aged users. Each of the users have their own style of using the application and have set their own expectations of it. A middle-aged or an aged user will be much calmer while using any application than someone who is young when it comes to the performance of the application. In general, we can say that mobile users have set incredibly high expectations of the applications available in the marketplace.

In the subsequent chapters, we will develop our understanding of how to overcome these challenges during automation design and how to strategize effectively. However, before that, we need to be aware of the different approaches for mobile test automation.