Creating the Android project

We will create a new Android project. This is done from the ASide menu by going to File | New Project. This then leads us through the wysiwyg guide to create a project.

In this particular case, we are using the following values for the required component names (clicking on the Next button in between screens):

• Application name: AndroidApplicationTestingGuide
• Company domain: blundell.com
• Form factor: Phone and Tablet
• Minimum SDK: 17
• Add an Activity: Blank Activity (go with default names)

The following screenshot shows the start of the form editor for reference:

When you click on Finish and the application is created, it will automatically generate the androidTest source folder under the app/src directory, and this is where you can add your instrumented test cases.

Tip

Alternatively, to create an androidTest folder for an existing Gradle Android project, you can select the src folder and then go to File | New | Directory. Then, write androidTest/java in the dialog prompt. When the project rebuilds, the path will then automatically be added so that you can create tests.

Package explorer

After having created our project, the project view should look like one of the images shown in the following screenshot. This is because ASide has multiple ways to show the project outline. On the left, we can note the existence of the two source directories, one colored green for the test source and the other blue for the project source. On the right, we have the new Android project view that tries to simplify the hierarchy by compressing useless and merging functionally similar folders.

Now that we have the basic infrastructure set up, it's time for us to start adding some tests, as shown in the following screenshot:

There's nothing to test right now, but as we are setting up the fundamentals of a Test-driven Development discipline, we are adding a dummy test just to get acquainted with the technique.

The src/androidTest/java folder in your AndroidApplicationTestingGuide project is the perfect place to add the tests. You could declare a different folder if you really wanted to, but we're sticking to defaults. The package should be the same as the corresponding package of the component being tested.

Right now, we are not concentrating on the content of the tests but on the concepts and placement of those tests.

Creating a test case

As described before, we are creating our test cases in the src/androidTest/java folder of the project.

You can create the file manually by right-clicking on the package and selecting New... | Java Class. However, in this particular case, we'll take advantage of ASide to create our JUnit TestCase. Open the class under test (in this case, MainActivity) and hover over the class name until you see a lightbulb (or press Ctrl/Command + 1). Select Create Test from the menu that appears.

These are the values that we should enter when we create the test case:

• Testing library: JUnit 3
• Class name: MainActivityTest
• Superclass: junit.framework.TestCase
• Destination package: com.blundell.tut
• Superclass: junit.framework.TestCase
• Generate: Select none

After entering all the required values, our JUnit test case creation dialog would look like this.

As you can see, you could also have checked one of the methods of the class to generate an empty test method stub. These stub methods may be useful in some cases, but you have to consider that testing should be a behavior-driven process rather than a method-driven one.

The basic infrastructure for our tests is in place; what is left is to add a dummy test to verify that everything is working as expected. We now have a test case template, so the next step is to start completing it to suit our needs. To do it, open the recently created test class and add the testSomething() test.

We should have something like this:

package com.blundell.tut;

import android.test.suitebuilder.annotation.SmallTest;

import junit.framework.TestCase;

public class MainActivityTest extends TestCase {

    public MainActivityTest() {
        super("MainActivityTest");
    }

    @SmallTest
    public void testSomething() throws Exception {
        fail("Not implemented yet");
    }
}

Tip

The no-argument constructor is needed to run a specific test from the command line, as explained later using am instrumentation.

This test will always fail, presenting the message: Not implemented yet. In order to do this, we will use the fail method from the junit.framework.Assert class that fails the test with the given message.

Test annotations

Looking carefully at the test definition, you might notice that we decorated the test using the @SmallTest annotation, which is a way to organize or categorize our tests and run them separately.

There are other annotations that can be used by the tests, such as:

Annotation	Description
@SmallTest	Marks a test that should run as part of the small tests.
@MediumTest	Marks a test that should run as part of the medium tests.
@LargeTest	Marks a test that should run as part of the large tests.
@Smoke	Marks a test that should run as part of the smoke tests. The android.test.suitebuilder.SmokeTestSuiteBuilder will run all tests with this annotation.
@FlakyTest	Use this annotation on the InstrumentationTestCase class' test methods. When this is present, the test method is re-executed if the test fails. The total number of executions is specified by the tolerance, and defaults to 1. This is useful for tests that may fail due to an external condition that could vary with time. For example, to specify a tolerance of 4, you would annotate your test with: @FlakyTest(tolerance=4).
@UIThreadTest	Use this annotation on the InstrumentationTestCase class' test methods. When this is present, the test method is executed on the application's main thread (or UI thread). As instrumentation methods may not be used when this annotation is present, there are other techniques if, for example, you need to modify the UI and get access to the instrumentation within the same test. In such cases, you can resort to the Activity.runOnUIThread() method that allows you to create any Runnable and run it in the UI thread from within your test: mActivity.runOnUIThread(new Runnable() { public void run() { // do somethings } });
@Suppress	Use this annotation on test classes or test methods that should not be included in a test suite. This annotation can be used at the class level, where none of the methods in that class are included in the test suite, or at the method level, to exclude just a single method or a set of methods.

Now that we have the tests in place, it's time to run them, and that's what we are going to do next.

Running the tests

There are several ways of running our tests, and we will analyze them here.

Additionally, as mentioned in the previous section about annotations, tests can be grouped or categorized and run together, depending on the situation.

Running all tests from Android Studio

This is perhaps the simplest method if you have adopted ASide as your development environment. This will run all the tests in the package.

Select the app module in your project and then go to Run | (android icon) All Tests.

If a suitable device or emulator is not found, you will be asked to start or connect one.

The tests are then run, and the results are presented inside the Run perspective, as shown in the following screenshot:

A more detailed view of the results and the messages produced during their execution can also be obtained in the LogCat view within the Android DDMS perspective, as shown in the following screenshot:

Running a single test case from your IDE

There is an option to run a single test case from ASide, should you need to. Open the file where the test resides, right-click on the method name you want to run, and just like you run all the tests, select Run | (android icon) testMethodName.

When you run this, as usual, only this test will be executed. In our case, we have only one test, so the result will be similar to the screenshot presented earlier.

Note

Running a single test like this is a shortcut that actually creates a run configuration for you that is specific to that one method. If you want to look into the details of this, from the menu, select Run | Edit Configurations, and under Android Tests, you should be able to see a configuration with the name of the test you just executed.

Running from the emulator

The default system image used by the emulator has the Dev Tools application installed, providing several handy tools and settings. Among these tools, we can find a rather long list, as is shown in the following screenshot:

Now, we are interested in Instrumentation, which is the way to run our tests. This application lists all of the packages installed that define instrumentation tag tests in their project. We can run the tests by selecting our tests based on the package name, as shown in the following screenshot:

When the tests are run in this way, the results can be seen through DDMS / LogCat, as described in the previous section.

Running tests from the command line

Finally, tests can be run from the command line too. This is useful if you want to automate or script the process.

To run the tests, we use the am instrument command (strictly speaking, the am command and instrument subcommand), which allows us to run instrumentations specifying the package name and some other options.

You might wonder what "am" stands for. It is short for Activity Manager, a main component of the internal Android infrastructure that is started by the System Server at the beginning of the boot process, and it is responsible for managing Activities and their life cycle. Additionally, as we can see here, it is also responsible for Activity instrumentation.

The general usage of the am instrument command is:

am instrument [flags] <COMPONENT> -r -e <NAME> <VALUE> -p <FILE>-w

This table summarizes the most common options:

Option	Description
-r	Prints raw results. This is useful to collect raw performance data.
-e <NAME> <VALUE>	Sets arguments by name. We will examine its usage shortly. This is a generic option argument that allows us to set the <name, value> pairs.
-p <FILE>	Writes profiling data to an external file.
-w	Waits for instrumentation to finish before exiting. This is normally used in commands. Although not mandatory, it's very handy, as otherwise, you will not be able to see the test's results.

To invoke the am command, we will be using the adb shell command or, if you already have a shell running on an emulator or device, you can issue the am command directly in the shell command prompt.

Running all tests

This command line will open the adb shell and then run all tests with the exception of performance tests:

$: adb shell 
#: am instrument -w com.blundell.tut.test/android.test.InstrumentationTestRunner
  
com.blundell.tut.MainActivityTest:

Failure in testSomething:

junit.framework.AssertionFailedError: Not implemented yet

at com.blundell.tut.MainActivityTest.testSomething(MainActivityTest.java:15)
at java.lang.reflect.Method.invokeNative(Native Method)
at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:191)
at android.test.AndroidTestRunner.runTest(AndroidTestRunner.java:176)
at android.test.InstrumentationTestRunner.onStart
                (InstrumentationTestRunner.java:554)
at android.app.Instrumentation$InstrumentationThread.run
                (Instrumentation.java:1701)

Test results for InstrumentationTestRunner=.F
Time: 0.002

FAILURES!!!
Tests run: 1,  Failures: 1,  Errors: 0

Note that the package you declare with –w is the package of your instrumentation tests, not the package of the application under test.

Running tests from a specific test case

To run all the tests in a specific test case, you can use:

$: adb shell 
#: am instrument -w -e class com.blundell.tut.MainActivityTest com.blundell.tut.test/android.test.InstrumentationTestRunner

Running a specific test by name

Additionally, we have the alternative of specifying which test we want to run in the command line:

$: adb shell 
#: am instrument -w -e class com.blundell.tut.MainActivityTest\#testSomething com.blundell.tut.test/android.test.InstrumentationTestRunner

This test cannot be run in this way unless we have a no-argument constructor in our test case; that is the reason we added it before.

Running specific tests by category

As mentioned before, tests can be grouped into different categories using annotations (Test Annotations), and you can run all tests in this category.

The following options can be added to the command line:

Option	Description
-e unit true	This runs all unit tests. These are tests that are not derived from InstrumentationTestCase (and are not performance tests).
-e func true	This runs all functional tests. These are tests that are derived from InstrumentationTestCase.
-e perf true	This includes performance tests.
-e size {small | medium | large}	This runs small, medium, or large tests depending on the annotations added to the tests.
-e annotation <annotation-name>	This runs tests annotated with this annotation. This option is mutually exclusive with the size option.

In our example, we annotated the test method testSomething() with @SmallTest. So this test is considered to be in that category, and is thus run eventually with other tests that belong to that same category, when we specify the test size as small.

This command line will run all the tests annotated with @SmallTest:

$: adb shell 
#: am instrument -w -e size small com.blundell.tut.test/android.test.InstrumentationTestRunner

Running tests using Gradle

Your gradle build script can also help you run the tests and this will actually do the previous commands under the hood. Gradle can run your tests with this command:

gradle connectedAndroidTest

Creating a custom annotation

In case you decide to sort the tests by a criterion other than their size, a custom annotation can be created and then specified in the command line.

As an example, let's say we want to arrange our tests according to their importance, so we create an annotation @VeryImportantTest, which we will use in any class where we write tests (MainActivityTest for example):

package com.blundell.tut;

/**
 * Marker interface to segregate important tests
 */
@Retention(RetentionPolicy.RUNTIME)
public @interface VeryImportantTest {
}

Following this, we can create another test and annotate it with @VeryImportantTest:

@VeryImportantTest
public void testOtherStuff() {
fail("Also not implemented yet");
}

So, as we mentioned before, we can include this annotation in the am instrument command line to run only the annotated tests:

$: adb shell 
#: am instrument -w -e annotation com.blundell.tut.VeryImportantTest com.blundell.tut.test/android.test. InstrumentationTestRunner

Running performance tests

We will be reviewing performance test details in Chapter 8, Testing and Profiling Performance, but here, we will introduce the available options to the am instrument command.

To include performance tests on your test run, you should add this command line option:

• -e perf true: This includes performance tests

Dry run

Sometimes, you might only need to know what tests will be run instead of actually running them.

This is the option you need to add to your command line:

• -e log true: This displays the tests to be run instead of running them

This is useful if you are writing scripts around your tests or perhaps building other tools.

Debugging tests

You should assume that your tests might have bugs too. In such a case, usual debugging techniques apply, for example, adding messages through LogCat.

If a more sophisticated debugging technique is needed, you should attach the debugger to the test runner.

In order to do this without giving up on the convenience of the IDE and not having to remember hard-to-memorize command-line options, you can Debug Run your run configurations. Thus, you can set a breakpoint in your tests and use it. To toggle a breakpoint, you can select the desired line in the editor and left-click on the margin.

Once it is done, you will be in a standard debugging session, and the debug window should be available to you.

It is also possible to debug your tests from the command line; you can use code instructions to wait for your debugger to attach. We won't be using this command; if you want more details, they can be found at (http://developer.android.com/reference/android/test/InstrumentationTestRunner.html).

Other command-line options

The am instrument command accepts other <name, value> pairs beside the previously mentioned ones:

Name	Value
debug	true. Set break points in your code.
package	This is a fully qualified package name of one or several packages in the test application.
class	A fully qualified test case class to be executed by the test runner. Optionally, this could include the test method name separated from the class name by a hash (#).
coverage	true. Runs the EMMA code coverage and writes the output to a file that can also be specified. We will dig into the details about supporting EMMA code coverage for our tests in Chapter 9, Alternative Testing Tactics.