Android testing framework

The instrumentation framework is the foundation of the testing framework. Instrumentation controls the application under tests and permits the injection of mock components required by the application to run. For example, you can create mock Contexts before the application starts and let the application use it.

All the interactions of the application with the surrounding environment can be controlled using this approach. You can also isolate your application in a restricted environment to be able to predict the results that force the values returned by some methods, or that mock persistent and unchanged data for the ContentProvider's databases or even the filesystem content.

A standard Android project has its instrumentation tests in a correlated source folder called androidTest. This creates a separate application that runs tests on your application. There is no AndroidManifest here as it is automatically generated. The instrumentation can be customized inside the Android closure of your build.gradle file, and these changes are reflected in the autogenerated AndroidManifest. However, you can still run your tests with the default settings if you choose to change nothing.

Examples of things you can change are the test application package name, your test runner, or how to toggle performance-testing features:

  testApplicationId "com.blundell.something.non.default"
  testInstrumentationRunner  "com.blundell.tut.CustomTestRunner"
  testHandleProfiling false
  testFunctionalTest true
  testCoverageEnabled true

Here, the Instrumentation package (testApplicationId) is a different package to the main application. If you don't change this yourself, it will default to your main application package with the .test suffix added.

Then, the Instrumentation test runner is declared, which can be helpful if you create custom annotations to allow special behavior; for example, each test runs twice upon failure. In the case of not declaring a runner, the default custom runner android.test.InstrumentationTestRunner is used.

At the moment, testHandleProfiling and testFunctionalTest are undocumented and unused, so watch out for when we are told what we can do with these. Setting testCoverageEnabled to true will allow you to gather code coverage reports using Jacoco. We will come back to this later.

Also, notice that both the application being tested and the tests themselves are Android applications with their corresponding APKs installed. Internally, they will be sharing the same process and thus have access to the same set of features.

When you run a test application, the Activity Manager (http://developer.android.com/intl/de/reference/android/app/ActivityManager.html) uses the instrumentation framework to start and control the test runner, which in turn uses instrumentation to shut down any running instances of the main application, starts the test application, and then starts the main application in the same process. This allows various aspects of the test application to work directly with the main application.

Gradle is an advanced build toolkit that allows you to manage dependencies and define a custom login to build your project. The Android build system is a plugin on top of Gradle, and this is what gives you the domain-specific language discussed previously such as setting a testInstrumentationRunner.

The idea of using Gradle is that it allows you to build your Android apps from the command line for machines without using an IDE such as a continuous integration machine. Also, with first line integration of Gradle into the building of projects in Android Studio, you get the exact same custom build configuration from the IDE or command line.

Other benefits include being able to customize and extend the build process; for example, each time your CI builds your project, you could automatically upload a beta APK to the Google play store. You can create multiple APKs with different features using the same project, for example, one version that targets Google play in an app purchase and another that targets the Amazon app store's coin payments.

Gradle and the Android Gradle plugin make for a powerful combination, and so, we will be using this build framework throughout the rest of the samples in this book.

During the evolution of your development project, your tests would be targeted to different devices. From simplicity, flexibility, and speed of testing on an emulator to the unavoidable final testing on the specific device you are intending your application to be run upon, you should be able to run your application on all of them.

There are also some intermediate cases such as running your tests on a local JVM virtual machine, on the development computer, or on a Dalvik virtual machine or Activity, depending on the case.

Every case has its pros and cons, but the good news is that you have all of these alternatives available to run your tests.

The emulator is probably the most powerful target as you can modify almost every parameter from its configuration to simulate different conditions for your tests. Ultimately, your application should be able to handle all of these situations, so it's much better to discover the problems upfront than when the application has been delivered.

The real devices are a requirement for performance tests, as it is somewhat difficult to extrapolate performance measurements from a simulated device. You will enjoy the real user experience only when using the real device. Rendering, scrolling, flinging, and other cases should be tested before delivering the application.