MobilePundits
www.mobilepundits.com
Introduction: You’ve developed a Mobile App...Now What? By the time you finish reading this, the mobile app landscape will have changed. Along with the development, testing mobile apps is quiet more challenging than testing based on desktop or web applications. Mobile app testing is an approach towards solving the emerging technological implications of Mobile Apps. iOS and Android have been the most successful operating systems so far. It is quite challenging to test and verify each and every application before releasing it for public use. In today’s business environment, testing seems to be a cost-effective solution to avoid any compromise on quality.
QA Challenges in Mobile Application Testing Variation in Device
Mobile app testing becomes difficult because of many operating systems and myriad devices. Compatibility is a major issue here because a mobile application can be deployed across multiple devices. They differ in: • Mobile operating systems like iOS, Android, BlackBerry, Windows, Symbian, etc. • Different versions of these operating systems such as iOS 4.x, 5.x, BB 4.x, 5.x, 6.x, Android 2.0, 2.x, 3.x, 4.x, Windows 7, Windows 8.
Mobile Testing Tool Availability The tools used for desktop and web-based applications do not work for mobile application. Hence, a complex scripting technique and new tool development is required for mobile application testing.
Industry Standards
Testing of mobile application must be performed according to certain industry standards so that the applications can be accepted globally by people using different mobile operating systems. This will help in making the application popular.
Necessity of Skilled QA specialist in Automation Testing
Every Quality team needs skilled quality assurance specialist not only to evaluate the automation tools that are present in the market for application testing but also to quickly recognize any bugs in the user interface as UI is very critical in the mobile world. A well-examined and fully functional mobile application can be easily be rejected by end users because of bad interface even though if it is working perfectly.
Cloud Computing
This technology offers a seamless web-based environment for mobile on simulators where the QA team can deploy, automate and test any mobile application. This approach of cloud computing has really helped in reducing the overall cost of a project under development.
Listed below are some of the most common problems that are faced by the testers in an environment of cloud computing: •High-cost •A Subscription Model •Lock-in •The Automation is time-consuming and image based •Automation cannot be done outside the framework
Mobile Testing Challenges for Native Applications
For many, “mobile application” means a native or a hybrid application. These are some commonly downloaded software applications from the respective app stores. The unique experience that they offer to the users improves the capabilities of the operating system and device for which they are developed. The downloaded is often monitored by the store, with mechanisms in place to charge consumers.
Though native applications offer rich user experience to the users – and plausible money spinning for the developers. Some complexity is added to the lives of those who test it. Testing is meant to make sure that the application can be successfully downloaded and executed on the device. To meet the challenges of the testing procedure, backward compatibility with each older device you are expected to support must also be checked.
Mobile Testing Challenges for Web Applications
Like any other website on internet, a mobile web app is also viewed by people around the world. Even when you are targeting people from one particular landscape or a network, it will help you understand dynamics of the world.
Mobile Application Testing Strategy The Test strategy should make sure that all the quality and performance guidelines are met. A few pointers in this area:
Selection of the devices Analyze the market and choose the devices that are widely used. (This decision mostly relies on the clients. The client or the app builders consider the popularity factor of a certain devices as well as the marketing needs for the application to decide what handsets to use for testing.)
Emulators The use of emulators is extremely useful in the initial stages of development, as they allow quick and efficient checking of the app. Emulator is a system that runs software from one environment to another environment without changing the software itself. It duplicates the features and work on real system. Types of Mobile Emulators
Device Emulator- provided by device manufacturers Browser Emulator- simulates mobile browser environments. Operating systems Emulator- Apple provides emulators for iPhones, Microsoft for Windows phones and Google Android phones
Some free and easy to use mobile device emulators are: iPhone Tester – All you need to do with this is – enter the URL in search box and you can see the real time preview of how it appears on an iPhone.
Mobile Phone Emulator – Used to test handsets like iPhone, blackberry, HTC, Samsung etc.
MobiReady– With this, not only can we test the web app, we can also check the code.
Responsivepx– It checks the responses of the web pages, appearances and functionality of the websites.
Screenfly– It is a customizable tool and used to test websites under different categories.
Mobile cloud Computing Services
The testing experts can efficiently make use of the environment of mobile cloud computing for deploying and testing the application. Using this technique, we can keep both the task and the data on the internet instead of keeping it on an individual device. This is what brings on-demand access in vogue. Lowering down the complexity of project implementation, organizations take the advantage of services of cloud computing so as to reduce the development and testing cost and boost the return on investment of a mobile software application solution.
Real Times Devices with Real Networks
It is quite natural that testing can be done against real networks. Real devices can be used at real locations for performing testing. This is something that the QA team just cannot avoid. Testing of real devices on real networks should always be an option whenever needed. This thing came into picture because every application will be used at different locations by end users who might use it from a remote area with inconsistent network signal strength.
Avoid Manual Work with Automation Tools
Automate whenever and wherever possible! Remote and Emulators, real time devices and solutions which support playback functionality and scripts help in saving and allow execution of more test cases with higher degree of consistency.
Any testing team must automate the task of mobile software applications so as to avoid any manual task. This will not only save cost but time as well. There are many tools available in the market for performing automation tools.
Types of Testing for a Mobile Application Usability Testing
This is the type of testing that includes visibility of text in the selected language, screen navigation, verification of online and offline functionality, feedback from interaction with the system, which means that the download app should be prompt with messages.
Compatibility Testing
Compatibility testing means validation of mobile application for myriad devices keeping in mind all the major operating systems, screen size and resolution as per the demand of users, proofreading if the integration server changes, editing of the mobile app isolation with other software apps on the device.
Interface Testing
Following are the things that integration testing covers: •Screens Resolution •Buttons •Text Inputs •Navigation flow like Facebook, Twitter, Reviews, Bookmarks, etc.
Service Testing
Validation of mobile software application not to behave as a server is included in this type of testing. It even covers checking of a service which can be used offline or takes too long or when a service goes down and responds back with malformed responses.
Performance Testing
Performance testing is something where you will need to verify the server connection of WIFI from 2G/3G or vice versa, size of the shared images used by the software code is in accordance with the desired response time, optimization of code with the CPU cycle, consumption of battery, memory leaks, certain resources like camera, GPS, etc., freed.
Operational Testing
This type of testing checks for the back-up of all the important information in the application, save and recovery plan if the battery goes down, if the data of mobile app is lost in the case of up-gradation from respective app store, accessibility of the app when user gets an alarm, message, call, reminder, etc. and power usage while accessing the application.
Low Level Resource Testing
Overuse of memory and not releasing is something that comes under low level resource testing. It also checks if temporary files of the applications are not cleaned, local database is growing too large. Low level testing will also see the garbage generated by the mobile app.
Security Testing
Security testing will include the encryption and decryption techniques used for communication of sensitive data, checking of multi-user support without any interference of the data between them, and checking of accessibility to files that are saved in the app by any unintended users and detect areas where tested application does not receive any nasty content.
Testing Recommendation For the types of testing mentioned it is a good idea to use some combination of real device and emulator testing, as recommended in the table below:
Enterprise applications must meet requirements for maintenance, administration and security. They “are more complex in functionality and architecture” and therefore it is important to test enterprise application on performance, security, and synchronization in addition to the standard functionality testing.
Test Automation Tools & Frameworks Testing mobile applications is traditionally done by manual execution of test cases and visual verification of the results. However, it is very time consuming. Using automation tools and testing frameworks yields quantifiable benefits and is recommended. Automated testing offers fast, repeatable and comprehensive test execution, including overnight or over weekend test runs. However the investment in tools, test script development and data collection is required and is and is not amenable to fast changing requirements. The development platform for mobile application which is a Personal Computer is different from the target platform which is a handheld device. It is impossible to test the various possible states a GUI can have. Because of these unlimited testing scenarios mobile GUI testing is more difficult compared to functional testing of desktop application.“Manual GUI testing is very error prone and hardly reproducible, and causes very high effort”.
Android Testing Framework The Android testing framework provides powerful tools for testing mobile applications. The following diagram summarizes the testing framework of Android:
The Android SDK (Software Development Kit) consists of tools for developing and testing android based mobile application. The Android SDK tools are available as Plug-in in Eclipse with ADT, and “in command-line form for use with other IDES”. The SDK“tools get information from the project of the application under test and use this information to automatically create the build files, manifest file, and directory structure for the test package”. The SDK also provides monkeyRunner, an API testing devices with Python programs, and UI/Application Exerciser Monkey, a command-line tool for stress-testing UIs by sending pseudo-random events to a device.
Sikuli Sikuli Test is a GUI testing framework that enables automation testing tasks.“It allows testers to write visual scripts to automate tests, to refer to GUI objects by their visual representation directly, and to provide robustness to changes in spatial arrangements of GUI components”. The script uses action statements to simulate the interactions and assertion statements to visually verify the outcomes of the interactions.
Test scripts under Sikuli Test are written to test traditional desktop GUI applications on Windows and Mac OSX, as well as mobile applications in an Android emulator and iOs simulators.
JaBUTi/ME: White Box Testing White box testing is a technique based on the internal structure of a given implementation, from which the test requirements are derived. In general, “white box testing criteria use a representation known as Control Flow Graph (CFG) to abstract the structure of the program or of part of the program, as a procedure or method”.
Server based testing JaBUTi/ME
JaBUTi (Java Bytecode Understanding and Testing Tool) is a complete tool suite for understanding and testing Java programs and Java-based components. JaBUTi differs “from other testing tools because it performs the static and analysis directly on Java Bytecode not on the Java source code”.
Mobile Test: Black Box Testing
An automatic black box testing tool for mobile devices introduced in this Chapter is Mobile Test. With Mobile Test building maintainable and reusable test cases for testing system level and application level software on various mobile devices is possible. Mobile Test observes the input and output information of an application. “From the input perspective, a mobile application receives two kinds of inputs“. The first input is form the user GUI such as keyboard events, touch events. The second input is environmental context events.
Mobile Test Architecture
Mobile Test architecture
The architecture of Mobile Test subdivides the system environment into layers to reduce the complexity the system. Each layer provides services to upper layers with the support of lower layers. In this way, the test control layer can be separated from the characteristics of the underlying devices.
List of available testing tools The table below gives a list of available testing frameworks with the platforms they can be tested on.
Available mobile automation testing tools and the platforms they run on
Fledge: BlackBerry device simulator that enables mobile application testing on a Personal Computer. Tests, various connectivity and state changes can be simulated.
MonkeyTalk (formerly known as FoneMonkey): offers the possibilities to save a script, load, read and modify it. FoneMonkey is designed to support developers and quality control tests. FoneMonkey automates testing on iOS simulators, Android emulators or real devices.
QTP(QuickTest Professional): is an automated testing tool provided by HP/Mercury Interactive. QTP uses VB scripting language to build its flows. It provides automated and regression testing and also generates test scripts that can be executed on local or remote mobile devices.
CTS (Compatibility test Suit): runs on Personal Computer and manages test execution. Individual test cases are executed on attached mobile devices or on an emulator.
UISpec: Behavior Driven Development framework for the iPhone that provides a full automated testing solution that drives the actual iPhone UI. It is modeled after the very popular RSpec for Ruby.
Hermes: black box automation tool for testing J2ME applications. Hermes supports application independence, and open interfaces for extensibility.
Eggplant: black box automation tool that runs on Mac OSX and Linux. It can test applications on a vast range of other platforms because it connects to and controls them using VNC (Virtual Network Computing).
Robotium: black box automation tool for Android Mobile Application. Robotium is a UI testing tool. The Framework provides API’s to test various kinds of Widgets/UI present in mobile applications developed with android SDK.
Conclusion Application development for mobile devices is evolving. The strategies presented in this thesis discuss the testing of mobile device applications and how important it is to plan a test strategy that is mobile-specific. Unique challenges like device challenges and software challenges of mobile devices need to be considered because traditional testing does not cover all characteristic important for mobile application. The use of automation test tools and test methods should be conducted for a successful testing result. The advantage of testing these apps before the deployment in a live environment cannot be stressed out. An app with bugs and errors will not only hurt the productivity but reputation as well.
About MobilePundits MobilePundits partners with global enterprises to drive their innovation-led growth. That's why Deloitte awarded MobilePundits among the top 100 most innovative companies. As a leading provider of next-generation consulting, technology and outsourcing solutions, MobilePundits helps clients across the world. Visit www.mobilepundits.com and see how MobilePundits, offers its best outsource and development services.
For more information, contact marketing@mobilepundits.com