Open Source Android Testing Tools

The shift towards mobile platforms is a strong trend currently and Android is the most widely adopted mobile OS with an estimated market share above 80% in 2014. You should naturally test all the apps developed for Android and a large number of open source testing tools and test automation frameworks have been developed to achieve this goal.

This article presents a list of open source Android testing tools. For each tool you will get a small description of its features and pointers to additional resources that discusses the tool more in details. Feel free to suggest any additional open source Android software testing framework or resource that you think might be included in this article. Tools are listed by alphabetical order. A list of open source Android testing tools projects that are not active is also included at the end of the article.

The open source Android software testing tools mentioned in this article are: Android Test Kit, AndroidX Test Libraries, Appium, AssertJ Android, AutomationCore, Barista, calabash-android, Cortado, Detox, Fastbot-Android, Green Coffee, Macaca, Magneto, MATE, Mobly, Monkey, Robolectric, RoboSpock, Robotium, UIAutomator, Selendroid, Spoon, Test Butler

Updates

April 11 2023: added AndroidX Test Libraries, Detox, Fastbot-Android
September 30 2019: added AutomationCore, MATE, Spoon
November 30 2017: updated Android Test Kit; removed AndroidJUnit4, NativeDriver; added Barista, Cortado, Green Coffee, Macaca, Mobly
January 24 2017: removed MonkeyTalk (acquired by Oracle); added AssertJ Android, Magneto, Test Butler

AndroidX Test Libraries

The AndroidX Test Library provides an extensive framework for testing Android apps. This library provides a set of APIs that allow you to quickly build and run test code for your apps, including JUnit 4 and functional user interface (UI) tests. You can run tests created using these APIs from the Android Studio IDE or from the command line.

Website: https://github.com/android/android-test

Android Test Kit

The Android Test Kit is the set of Google open source testing tools for Android. This library provides a set of APIs that allow you to quickly build and run test code for your apps, including JUnit 4 and functional user interface (UI) tests. You can run tests created using these APIs from the Android Studio IDE or from the command line.

Website: https://google.github.io/android-testing-support-library/
Additional resources
* Android application testing with the Android test framework – Tutorial
* Espresso for Android is here!

Appium

Appium is an open source test automation framework for use with native and hybrid mobile apps. Appium drives iOS and Android apps using the WebDriver JSON wire protocol. Appium’ philosophy is that testing native apps shouldn’t require including an SDK or recompiling your app. Appium aims to automate any mobile app from any language and any test framework, with full access to back-end APIs and DBs from test code. You can write tests with your favorite development tools with the Selenium WebDriver API and language-specific client libraries.

Website: http://appium.io/

Additional resources
* Android UI testing with Appium

AssertJ Android

AssertJ Android is a set of AssertJ assertions geared toward testing Android. Writing tests is not the most glamorous part of developing an Android application but it is an invaluable one. Using libraries like JUnit and AssertJ provide a great starting point for writing tests. This library is an extension of AssertJ which aims to make it even easier to test Android.

Website: http://square.github.io/assertj-android/
http://joel-costigliola.github.io/assertj/

AutomationCore

AutomationCore provides a complete testing platform for UI (Web, iOS, Android, Win) and API testing. It provides a feature rich and viable testing solution for end to end testing. They’re designed to be fast, scalable, reliable and adaptable to any requirements for ever growing projects. AutomationCore has a modular design. Each project/component is treated as a module, fully capable of interacting with one another. This allows for multi component and multiplatform testing. Eg. Create user through component A (API), validate in component B (web), do action in component C (Android), validate results in component D (iOS)

Website: https://github.com/autonomx/Autonomx

Barista

Barista makes developing UI test faster, easier and more predictable. Built on top of Espresso, it provides a simple an discoverable API, removing most of the boilerplate and verbosity of common Espresso tasks. You and your Android team will write tests with no effort.

Website: https://github.com/SchibstedSpain/Barista

Calabash-android

Calabash enables you to write and execute. Calabash is an open source tool to write automated acceptance tests of mobile apps that supports Android and iOS native apps. Calabash consists of libraries that enable test-code to programmatically interact with native and hybrid apps. The interaction consists of a number of end-user actions. Calabash could be compared to Selenium WebDriver. However, it is important to realize that interacting with a web app from a desktop computer is vastly different than interacting with a native app using a touch screen. Calabash provides APIs that are specialized to native apps running on touch screen devices.

Website: http://calaba.sh/, https://github.com/calabash/calabash-android

Additional resources
* A better way to test Android applications using Calabash
* Calabash Android: query language basics

Cortado

Cortado is an open source software testing framework for Android that provides a layer of abstraction above Espresso, so it’s a bit easier to use. As there is Google Espresso underneath, you can still mess up your tests the same way you would when using pure Espresso. Cortado just gives you a bit nicer way to communicate with Espresso.

Website: https://github.com/blipinsk/cortado

Detox

Detox is an open source gray box end-to-end testing and automation framework for mobile apps. High velocity native mobile development requires us to adopt continuous integration workflows, which means our reliance on manual QA has to drop significantly. Detox tests your mobile app while it’s running in a real device/simulator, interacting with it just like a real user. The most difficult part of automated testing on mobile is the tip of the testing pyramid – E2E. The core problem with E2E tests is flakiness – tests are usually not deterministic. We believe the only way to tackle flakiness head on is by moving from black box testing to gray box testing. That’s where Detox comes into play.

• Cross Platform: Write cross-platform end-to-end tests in JavaScript. Currently supports iOS and Android.
• Debuggable: Modern async-await API allows breakpoints in asynchronous tests to work as expected.
• Automatically Synchronized: Stops flakiness at the core by monitoring asynchronous operations in your app.
• Made For CI: Execute your E2E tests on CI platforms like Travis CI, Circle CI or Jenkins without grief.
• Runs on Devices: Gain confidence to ship by testing your app on a device/simulator just like a real user (not yet supported on iOS).
• Test Runner Agnostic: Detox provides a set of APIs to use with any test runner without it. It comes with Jest integration out of the box.

Websites: https://github.com/wix/Detox, https://wix.github.io/Detox/

Fastbot-Android

Fastbot-Android is an open source model-based testing tool for modeling GUI transitions to discover app stability problems. It combines machine learning and reinforcement learning techniques to assist discovery in a more intelligent way. Fastbot is compatible with multiple Android OS systems, including original Android, Android 5-13 and a variation of modified Andriod-based system by domestic manufacturers. Inherited from original Monkey, Fastbot allows for fast action insertion as high as 12 actions per second. Expert system is equipped with the ability to customize deeply based on needs from different business lines. Fastbot is a model-based-testing tool. Model is build via graph transition with the consideration of high reward choice selection. Fastbot supports non-standard widgets by computer vision techniques such as YOLOv3, ocr and cv segmentation.

Website: https://github.com/bytedance/Fastbot_Android

Green Coffee

Green Coffee is an open source Android library that allows you to run your acceptance tests written in Gherkin in your Android instrumentation tests using the step definitions that you declare. Features are written using the Gherkin language. Each feature consists of one or more scenarios that describe different situations in order to test that feature. Each scenario consists of steps that will simulate user interactions with the UI.

Website: https://github.com/mauriciotogneri/green-coffee

Macaca

Macaca is an open-source automation test solution for native, hybrid, mobile web and web application on mobile and desktop platforms. Macaca is “cross-platform”. It means you can use the same API to write test scripts, and same test scripts to test your apps running on devices such as iOS, Android or desktops. Macaca doesn’t limit what languages you will use to write your tests.

Website: https://github.com/alibaba/macaca

Magneto

Magneto is an open source test automation framework that allows to write smart and powerful tests for Android apps. Magneto was built by Automation Engineers for Automation Engineers out of necessity for a mobile centric test automation framework that’s easy to setup, run and utilize. Magneto is written in Python for Android devices. It utilizes the uiautomator tool via a Python wrapper and pytest as a test framework. Magneto can be triggered from CLI, IDE and CI.

Website: http://getmagneto.com/

MATE

MATE is an open source tool for automated android app testing featuring various (genetic) algorithms. The project is divided into two parts: the MATE framework and unit tests for android managed in this repository and a server that runs on a host machine which is responsible for some analysis that can not be performed by the instrumentation tests themselves.

Website: https://github.com/mate-android-testing/mate

Mobly

Mobly is a Python-based test framework that specializes in supporting test cases that require multiple devices, complex environments, or custom hardware setups. Mobly can support many different types of devices and equipment, and it’s easy to plug your own device or custom equipment/service into Mobly.

Web site: https://github.com/google/mobly and https://github.com/google/mobly-bundled-snippets

Monkey

The Monkey is a open source software testing tool developed by Google that runs on your emulator or device and generates pseudo-random streams of user events such as clicks, touches, or gestures, as well as a number of system-level events. You can use the Monkey to stress-test your apps in a random yet repeatable manner. The Monkey is a command-line tool that that you can run on any emulator instance or on a device. It sends a pseudo-random stream of user events into the system, which acts as a stress test on the application software you are developing. Monkey generates events and sends them to the system. It also watches the system under test and looks for three conditions, which it treats specially.

Web site: http://developer.android.com/tools/help/monkey.html

Robolectric

Robolectric is a unit test framework that de-fangs the Android SDK jar so you can test-drive the development of your Android app. Tests run inside the JVM on your workstation in second. Robolectric handles inflation of views, resource loading, and lots of other stuff that’s implemented in native C code on Android devices. This allows tests to do most things you could do on a real device. It’s easy to provide our own implementation for specific SDK methods too. Robolectric lets you run your tests on your workstation, or on your Continuous Integration environment in a regular JVM, without an emulator.

Web site: http://robolectric.org/
Additional resources
* Better Android Testing with Robolectric 2.0
Using Robolectric for Android testing – Tutorial

RoboSpock

RoboSpock is an open source test framework for Android that provides an easy way of writing Behavior Driven Development (BDD) specifications with Groovy language and supports the Google Guice library. It combines two main solutions in the area: Robolectric and Spock framework.

Web site: http://robospock.org/
Additional resources
* RoboSpock – Behavior Driven Development (BDD) for Android

Robotium

Robotium is an Android test automation framework that has full support for native and hybrid applications. Robotium makes it easy to write powerful and robust automatic black-box UI tests for Android applications. With the support of Robotium, test case developers can write function, system and user acceptance test scenarios, spanning multiple Android activities.

Web site: https://code.google.com/p/robotium/
Additional resources
* Robotium – Testing Android User Interface
* Android user interface testing with Robotium – Tutorial

UIAutomator

The UIAutomator testing framework lets you test your user interface (UI) efficiently by creating automated functional UI test cases that can be run against your app on one or more devices.

Web site: http://developer.android.com/tools/help/uiautomator/index.html
Additional resources
* Automatic Android Testing with UiAutomator

Selendroid

Selendroid is a test automation framework that drives off the UI of Android native and hybrid applications (apps) and the mobile web. Tests are written using the Selenium 2 client. Selendroid can be used on emulators and real devices and can be integrated as a node into the Selenium Grid for scaling and parallel testing.

Website: http://selendroid.io/
Additional resources
* Mobile Test Automation with Selendroid
* Road to setup Selendroid and create first test script of android application
* Up and running with: Selendroid

Spoon

Spoon aims to simplify testing Android applications by distributing instrumentation test execution and displaying the results in a meaningful way. Instead of attempting to be a new form of testing, Spoon makes existing instrumentation tests more useful. Using the application APK and instrumentation APK, Spoon runs the tests on multiple devices simultaneously. Once all tests have completed a static HTML summary is generated with detailed information about each device and test. In addition to simply running instrumentation tests, Spoon has the ability to snap screenshots at key points during your tests which are then included in the output. This allows for visual inspection of test executions across different devices.

Website: https://github.com/square/spoon

Test Butler

Test Butler is an open source Android testing tool developed by LinkedIn. Test Butler is a two-part project. It includes an Android library that your test code can depend on, as well as a companion Android app apk that must be installed on your Android emulator before using Test Butler. You can build the Test Butler APK yourself from source, or download the binary from Bintray.

Website: https://github.com/linkedin/test-butler
Additional resources
* Open Sourcing Test Butler, Reliable Android Testing, at Your Service
* “Your Espresso, Sir!”, or Stepping Up Your UI Tests With Test Butler

Some discontinued projects of Android testing tools

A list of some seemingly discontinued open source projects (or at least not updated in recent months) for Android testing tools that you might want to check… and maybe taking back to life.

Emmagee

Emmagee is a practical, handy performance test tool for specified Android App, which can monitor CPU, memory and network traffic. Additionally, it also provides several cool features such as customizing interval of collecting data, rendering real-time process status in a floating window, and much more.

Web site: https://github.com/kevinkong/Emmagee

NativeDriver

NativeDriver is an implementation of the WebDriver API that drives of the UI of a native application rather than a web application.

Web site: https://github.com/sonixlabs/nativedriver

Sirocco

Scirocco is an open source Application testing tool from Sonix. You will be able to access necessary testing devices from Eclipse. Instead of hand operation, Scirocco provides automatic Android application’s testing functionalities. It supports Google NativeDriver and AndroidDriver as main testing library.

Web site: https://github.com/sonixlabs/scirocco-webdriver