Monday, June 23, 2014

Future Trends in Mobile Test Automation Tools


In Mobile test Automation tools now we see a trend of these test automation tools moving towards a combination of Object & Image comparison based approach and calling it hybrid approach of mobile test automations. Pre-dominantly Image comparison based tools have started offering Object comparison based testing as well and Tools like mAutomate are moving towards image comparison, by virtue of which mAutomate is now able to provide UI verification along with functional test execution.

Looking at this current trend, I can predict that trend of mobile test automation tools moving towards advance hybrid approach of test automation enabling automatic system/UI object mapping, automatic transversal of apps, Seamless UI verifications, automatic Security, random interrupt and app store compliance certification in near future.

Thursday, June 5, 2014

Evolution of Mobile Test Automation Tools

With the evolution of mobile phones platforms from basic feature phones based on J2me/BREW to current date smartphones powered by Android, iOS and WP8, the mobile applications have evolved along with from simple few screen interactions to complex integrated business solutions.

In the feature phone day’s as the app’s were smaller in size and were built as few screen interactions to solve a particular business problem all the functional testing was done manually. As the underlining business logic started becoming complex, some of the developers started writing unit test scripts for the mobile app business logic classes & methods using the frameworks derived from JUnit such as JM Unit, B Unit and OC Unit. These unit test scripts were checked-in into the source control with the source code and integrated with continuous integration (CI) systems, to execute automatically on every built.
As the complexity and size of the mobile applications increased with device capabilities and processing power, the apps require testing of large number of test cases that cannot be tested manually in every release cycle and this created a need of a UI test automation for mobile applications. This need gave birth to Image comparison (OCR based tools) powered by VnC based tools and hardwired mobile phone dock’s with camera’s mounted over the device screens. These tools provided basic record and playback capabilities, with initial scripting support based on manually created on-screen Image Object mapping repo’s. As these tools had the capabilities of pushing key events and relaying the mobile screen content on a connected computer for execution of automation scripts, they are also used for testing automation of mobile application on remotely available device connected over internet i.e. the services like device-anywhere.

With the increase in popularity of iOS & Android platforms, people started using Instruments support provided by the iOS platform SDK xcode and Robotium for Android. Testing with Instruments required developing unit test type ‘scripts’ and executing then on emulators or usb/wifi connected devices. QA engineers have to often revert to development team for modification of these scripts to represent the modified or required new test scenarios as none of these frameworks provided the capabilities of recording the test scripts. These frameworks were able to fulfill the need of functional test automation for the mobile apps, but were not able to support with the need of UI test automation or execution of test scripts on remote devices in different geographies and operator networks.

Next came in the wave of Object comparison based test automation tool for mobile application; these tools require instrumentation of the application source code with their automation library. This library captures all the application objects, their properties and user actions at the time of recording and replays user actions at the time of execution to assert the object properties and give results. Some of this of tools allow to execute test scripts only on USB/wifi connected devices, while some others provides the capabilities execute test scripts on remote devices connected over internet and hosted anywhere across the globe. Few of them allow simulating desired environmental conditions at the time of recording and execution along with senor data simulation. They also provide features like adding assertions and regressions and editing of recorded test scripts for advance users.