Android Application Programming with OpenCV

Chapter 1. Setting Up OpenCV

This chapter is a quick guide for setting up a development environment for Android and OpenCV. We will also look at the OpenCV sample applications, documentation, and community.

By the end of this chapter, our development environment will include the following components:

Java Development Kit (JDK) 6: It includes tools for Java programming.
Cygwin 1.7 or greater (Windows only): It is a compatibility layer that provides Unix-like programming tools on Windows.
Android Software Development Kit (Android SDK) r21.0.1 or greater: It includes tools for programming Android apps in Java.
Android Native Development Kit (Android NDK) r8d or greater: It includes tools for programming Android apps in C++. Although this book deals with Java programming, OpenCV also includes Android-compatible C++ samples that you may want to compile and run.
Eclipse 4.2.1 (Juno) or greater: It is an integrated development environment (IDE).
Java Development Tools (JDT): It is an Eclipse plugin for Java programming (already included in most Eclipse distributions).
C/C++ Development Tooling (CDT) 8.1.1 or greater: It is an Eclipse plugin for C/C++ programming.
Android Development Tools (ADT) 21.0.1 or greater: It is an Eclipse plugin for Android programming.
OpenCV4Android 2.4.3.2 or greater: It includes OpenCV's Android version, including Java and C++ libraries.

There are many possible ways to install and configure these components. We will cover several common setup scenarios, but if you are interested in even more options, see OpenCV's official documentation at http://docs.opencv.org/doc/tutorials/introduction/android_binary_package/O4A_SDK.html.

Setting up a development environment

Basically, we have a choice among the following approaches:

Install a prepackaged, preconfigured development environment that contains all the components we need.
Install various components separately and configure them to work together. Within this approach, we may do either of the following:
- Use a prepackaged, preconfigured version of OpenCV
- Configure and build OpenCV from source

Let's look at each of these alternatives in detail.

Getting a ready-made development environment – Tegra Android Development Pack (TAPD)

Tegra Android Development Pack (TADP) contains a complete, preconfigured development environment for Android, OpenCV, and some other libraries. TADP builds apps that are optimized for NVIDIA's Tegra processors. Despite being optimized for Tegra, the apps are compatible with other hardware too.

Tip

If you are setting up an Android development environment from scratch, I recommend TADP. It contains recent versions of all our required software and its setup process is simple.

TADP also contains some extras that we do not require for this book. For a complete list of TADP's contents, see the official description at https://developer.nvidia.com/tegra-android-development-pack.

To set up TADP, we just need to download and install it from a secure section of NVIDIA's website. Here are the required steps:

Join the NVIDIA Registered Developer Program at https://developer.nvidia.com/user/register. (It is free.)
Log in at https://developer.nvidia.com/user/login.
Complete your user profile at https://developer.nvidia.com/user/me/profile/rdp_profile.
Apply to join the Tegra Registered Developer Program at https://developer.nvidia.com/rdp/applications/tegra-registered-developer-program. (It is free, too!) Wait for NVIDIA to send you an acceptance email. Normally, you might receive it a few minutes after applying.
Go to https://developer.nvidia.com/tegra-resources and find the download link for TADP's latest version. At the time of writing, the latest version is 2.0r2. There are installers for Windows (32-bit or 64-bit), Mac, and Ubuntu (32-bit or 64-bit). Download and run the appropriate installer.
When the installer presents the Installation Directory step, we can enter any destination, which we will refer to as <tadp>. By default, <tadp> is C:\NVPACK (Windows) or ~/NVPACK (Mac and Ubuntu).
When the installer presents the Installation Options step, we may select any option: Complete, Express, or Custom. Compared to an Express installation, a Complete or Custom installation may include additional versions of Android SDK and binary images of Tegra Android OS, which is NVIDIA's customization of Android. If in doubt, choose Express.
When the installer presents the Proxy Configuration step, we may leave all fields blank unless we are using a proxy server.
After finishing all of the installer's configuration steps, wait for TADP's content to be downloaded and installed.

That's all! Before proceeding, let's just take a note of the locations where TADP has installed certain components. For TADP 2.0r2 (the latest version at the time of writing), the locations are as follows:

Android SDK is located at <tadp>/android-sdk-macosx. We will refer to this location as <android_sdk>.
Android NDK is located at <tadp>/android-ndk-r8d. We will refer to this location as <android_ndk>.
OpenCV4Android is located at <tadp>/OpenCV-2.4.3.2-android-sdk-tadp. We will refer to this location as <opencv>.
Eclipse is located at <tadp>/eclipse. We will refer to this location as <eclipse>.

Note

The TADP installer automatically edits the system's PATH to include <android_sdk>/platform-tools and <android_sdk>/tools. Also, it creates an environment variable called NDKROOT, whose value is <android_ndk>.

Now, we can proceed to Building the OpenCV Samples with Eclipse, later in this chapter.

Tip

Downloading the example code

You can download the example code files for all Packt books you have purchased from your account at http://www.packtpub.com. If you purchased this book elsewhere, you can visit http://www.packtpub.com/support and register to have the files e-mailed directly to you. The example code for this book is also available from the author's website at http://nummist.com/opencv/.

Assembling a development environment piece-by-piece

Instead of using TADP as a ready-made solution, we may assemble our own development environment. Broadly, this task has two stages:

Set up a general-purpose Android development environment
Set up OpenCV for use in this environment

Let's start by looking at the setup steps for a general-purpose Android development environment. We will not delve into very much detail here, because good instructions are available at the given links and, because you, as an Android developer, have probably been through similar steps before.

Tip

If you already have an Android development environment and you just want to add components to it, some of the following steps will not apply to you.

Here are the steps:

If we are using Windows or Linux, we may need to obtain JDK 6 manually. (On Mac, if JDK 6 is not present, the operating system will automatically offer to install it when needed.) The JDK 6 installers or packages are available for Windows and many Linux distributions at http://www.oracle.com/technetwork/java/javase/downloads/jdk6downloads-1902814.html. Alternatively, on Linux, check your repository for the JDK packages. Install JDK 6.
Download Eclipse and unzip it to any destination, which we will refer to as <eclipse>. There are many versions from which we may choose. Google provides an Eclipse distribution called Android Developer Tools (ADT) Bundle, which comes with Android SDK and the ADT plugin prepackaged and preconfigured. ADT Bundle is available at http://developer.android.com/sdk/index.html. Many other up-to-date Eclipse distributions are available at http://www.eclipse.org/downloads/. Of these, Eclipse for Mobile Developers is a good and minimalist choice as a foundation for an Android development environment.
If we did not get the ADT Bundle, we now need to set up Android SDK and the ADT plugin for Eclipse. Go to http://developer.android.com/sdk/index.html and get the download named Android SDK Tools. Install or unzip it to any destination, which we will refer to as <android_sdk>. Open Eclipse and install the ADT plugin according to the official instructions at http://developer.android.com/sdk/installing/installing-adt.html. Restart Eclipse. A window, Welcome to Android Development, should appear. Click on Use Existing SDKs, browse to <android_sdk>, and click on Next. Close Eclipse.
If we are using Windows, download and install Cygwin from http://cygwin.com/install.html.
Download Android NDK from http://developer.android.com/tools/sdk/ndk/index.html. Unzip it to any destination, which we will refer to as <android_ndk>.
Edit your system's PATH to include <android_sdk>/platform-tools and <android_sdk>/tools. Also, create an environment variable named NDKROOT with the value as <android_ndk>. (If you are unsure how to edit PATH and other environment variables, see Appendix A: Editing environment variables.)
Tip
Editing environment variables on Windows
The system's Path variable and other environment variables can be edited in the Environment Variables window of Control Panel.
On Windows Vista/7/8, open the Start menu and launch Control Panel. Now, go to System and Security | System | Advanced system settings. Click on the Environment Variables button.
On Windows XP, open the Start menu and go to Control Panel | System. Click on the Advanced tab. Click on the Environment Variables button.
Now, under System variables, select an existing environment variable, such as Path, and click on the Edit button. Alternatively, make a new environment variable by clicking on the New button. Edit the variable's name and value as needed. For example, if we want to add C:\android-sdk\platform-tools and C:\android-sdk\tools to Path, we should append ;C:\android-sdk\platform-tools;C:\android-sdk\tools to the existing value of Path. Note the use of semicolons as separators.
To apply the changes, click on all the OK buttons until we are back in the main window of Control Panel. Now, log out and again log in.
Editing environment variables on Mac
Edit ~/.profile.
To append to an existing environment variable, add a line such as export PATH=$PATH:~/android-sdk/platform-tools:~/android-sdk/tools. This example appends ~/android-sdk/platform-tools and ~/android-sdk/tools to PATH. Note the use of colons as separators.
To create a new environment variable, add a line such as export NDKROOT=~/android-ndk.
Save your changes, log out, and again log in.
Editing environment variables on Ubuntu
Edit ~/.pam_environment.
To append to an existing environment variable, add a line such as PATH DEFAULT=${PATH}:~/android-sdk/platform-tools:~/android-sdk/tools. This example appends ~/android-sdk/platform-tools and ~/android-sdk/tools to PATH. Note the use of colons as separators.
To create a new environment variable, add a line such as NDKROOT DEFAULT=~/android-ndk.
Save your changes, log out, and again log in.

Now, we have an Android development environment but we still need OpenCV. We may choose to download a prebuilt version of OpenCV or we may build it from source. These options are discussed in the following two subsections.

Tip

Generally, Android applications should use a prebuilt version of OpenCV. One important reason is that the prebuilt versions are available for Android users as shared libraries, which save disk space and simplify updates.

For the purpose of this book's project, there is no need to build OpenCV from source. We just mention this option for completeness, since it may be of interest to advanced users who want to modify OpenCV.

Getting the prebuilt OpenCV4Android

The prebuilt versions of OpenCV4Android can be downloaded from http://sourceforge.net/projects/opencvlibrary/files/opencv-android/. Look for files that have opencv-android in the name, such as OpenCV-2.4.5-android-sdk.zip (the latest version at the time of writing). Download the latest version and unzip it to any destination, which we will refer to as <opencv>.

Building OpenCV4Android from source

Alternatively, the process for building OpenCV4Android from trunk (the latest, unstable source code) is documented at http://code.opencv.org/projects/opencv/wiki/Building_OpenCV4Android_from_trunk. For a summary of the process for building from trunk, continue reading this section. Otherwise, skip ahead to Building the OpenCV samples with Eclipse, later in this chapter.

Tip

Since trunk contains the latest and unstable source code, there is no guarantee that the build process will succeed. You may need to do your own troubleshooting if you want to build from trunk.

To build OpenCV from source, we need the following additional software:

Git: It is a Source Control Management (SCM) tool, which we will use to obtain OpenCV's source code. On Windows or Mac, download and install Git from http://git-scm.com/. On Linux, install it using your package manager. For example, on Ubuntu, open Terminal and run $ sudo apt-get install git-core.
CMake: It is a set of build tools. On Windows or Mac, download and install CMake from http://www.cmake.org/cmake/resources/software.html. On Linux, install it using your package manager. For example, on Ubuntu, open Terminal and run $ sudo apt-get install cmake.
Apache Ant 1.8.0 or greater: It is a set of build tools for Java. On Linux, just install Ant using your package manager. For example, on Ubuntu, open Terminal and run $ sudo apt-get install ant. On Windows or Mac, download Ant from http://ant.apache.org/bindownload.cgi and unzip it to any destination, which we will refer to as <ant>. Make the following changes to your environment variables:
- Add <ant>/bin to PATH.
- Create a variable, ANT_HOME, with the value <ant>.
Python 2.6 or greater (but not 3.0 or greater): It is a scripting language that is used by some of the OpenCV build scripts. An appropriate version of Python comes preinstalled on Mac and most Linux systems, including Ubuntu. On Windows, download and install Python from http://www.python.org/getit/. If you have installed multiple versions of Python on your system, ensure that an installation of Python 2.6 or greater (but not 3.0 or greater) is the only one in Path (Windows) or PATH (Mac, Linux, or other Unix-like systems). The OpenCV build scripts do not run properly with Python 3.0 or greater.

Once we have these prerequisites, we may download the OpenCV source code to any location, which we will refer to as <opencv_source>. Then, we may build it using an included script. The steps are platform-specific, and are described as follows:

On Windows, copy <opencv>\android\scripts\wincfg.cmd.tmpl to <opencv>\android\scripts\wincfg.cmd. Edit <opencv>\android\scripts\wincfg.cmd. The locations of several of the prerequisites are declared in this file. Modify them so that they are correct for your system. Save your changes. Then, open Git Bash (Git's command prompt) and run the following commands:

$ git clone git://code.opencv.org/opencv.git <opencv_source>
$ cd <opencv_source>/android
$ scripts/cmake_android.cmd
$ cd build
$ make -j8

On Mac, Ubuntu, or other Unix-like systems, open Terminal (or another command line shell) and run the following commands:

$ git clone git://code.opencv.org/opencv.git <opencv_source>
$ cd <opencv_source>/android
$ sh ./scripts/cmake_android.sh
$ cd build
$ make -j8

If all goes well, we should get a build of OpenCV4Android in <opencv_source>/android/build. We may move it elsewhere if we wish. We will refer to its final location as <opencv>.

Building the OpenCV samples with Eclipse

Building and running a few sample applications is a good way to test that OpenCV is correctly set up. At the same time, we can practice using Eclipse.

Let's start by launching Eclipse. The Eclipse launcher should be located at <eclipse>/eclipse.exe (Windows), <eclipse>/Eclipse.app (Mac), or <eclipse>/eclipse (Linux). Run it.

We should see a window called Workspace Launcher, which asks us to select a workspace. A workspac e is the root directory for a set of related Eclipse projects. If we are using TADP, enter <tadp>/nvsample_workspace, which is a workspace where the OpenCV4Android library, samples, and tutorials are already set up as projects. Otherwise, enter any location you choose.

Tip

We can return to Workspace Launcher anytime via the menu: File | Switch Workspace | Other….

If the Welcome to Eclipse screen appears, click on the Workbench button.

Now, we should see a window with several panels, including Package Explorer. If we are not using TAPD, we need to import the OpenCV sample projects into our new workspace. Right-click on Package Explorer and select Import… from the context menu.

The Import window should appear. Select General | Existing Projects into Workspace, and then click on Next>.

On the second page of the Import window, enter <opencv> in the Select root directory: field. Under the Projects: label, a list of detected projects should appear (If not, click on Refresh). The list should include OpenCV library, samples, and tutorials. Ensure that all projects are selected and click on Finish to import them.

Once the projects are imported, we may need to fix some configuration issues. Our development environment may have different paths, and different versions of the Android SDK, than the ones in the samples' default configuration.

Any resulting errors will be reported in the Problems tab.

Tip

We should start by resolving any errors in the OpenCV Library project, as the samples and tutorials depend on the library.

The following are some of the common configuration problems, and their symptoms and solutions:

The target Android version might not be properly specified. The symptoms are that imports from the java and android packages fail, and there are error messages such as The project was not built since its build path is incomplete. The solution is to right-click on the project in Package Explorer, select Properties from the context menu, select the Android section, and checkmark one of the available Android versions. These steps should be repeated for all projects. At compile time, OpenCV and its samples must target Android 3.0 (API level 11) or greater, though at runtime they also support Android 2.2 (API level 8) or greater.
If we are working on Mac or Linux, the C++ samples might be misconfigured to use the Windows build executable. The symptom is an error message such as Program "{ndk}/ndk-build.cmd" not found in PATH. The solution is to right-click on the project in Package Explorer, select Properties from the context menu, select the C/C++ Build section, and edit the Build command: field to remove the .cmd extension. These steps should be repeated for all the native (C++) projects, which include OpenCV Sample - face-detection and OpenCV Tutorial 2 - Mixed Processing.
Tip
If we are using the projects in TADP's <tadp>/nvsample_workspace, we only need to troubleshoot the projects that have names starting with OpenCV. For this book's purposes, the other TADP samples are not relevant.

Once the OpenCV projects no longer show any errors, we can prepare to test them on an Android device. Recall that the device must have Android 2.2 (Froyo) or a greater version, and a camera. To enable Eclipse to communicate with the device, we must enable the device's USB debugging option with the help of the following steps:

Open the Settings app.
On Android 4.2 or greater, go to the About phone or About tablet section and tap Build number seven times. This step enables the Developer options section.
Go to the Developer options section (on Android 4.0 or greater) or the Applications | Development section (on Android 3.2 or less). Enable the USB debugging option.

Now, open the Play Store app, and find and install the OpenCV Manager app. (The app's page in the Play Store should look similar to the previous screenshot.) OpenCV Manager takes care of checking for any OpenCV library updates when we run any OpenCV applications.

Tip

If you do not have the Play Store app on your device, then you need to install OpenCV Manager and certain OpenCV libraries via USB as per the instructions at http://docs.opencv.org/android/service/doc/UseCases.html.

Now, we must prepare our main computer for communication with the Android device. The required steps vary, depending on our operating system.

On Windows, we need to install the proper USB drivers for the Android device. Different vendors and devices have different drivers. The official Android documentation provides links to the various vendors' driver download sites at http://developer.android.com/tools/extras/oem-usb.html#Drivers.

On Linux, before connecting an Android device via USB, we must specify the device's vendor in a permissions file. Each vendor has a unique ID number, as listed in the official Android documentation at http://developer.android.com/tools/device.html#VendorIds. We will refer to this ID number as <vendor_id>. To create the permissions file, open a command prompt application (such as Terminal) and run the following commands:

$ cd /etc/udev/rules.d/
$ sudo touch 51-android.rules
$ sudo chmod a+r 51-android-rules

Note that the permissions file needs to have root ownership, so we use sudo when creating or modifying it. Now, open the file in an editor such as gedit:

$ sudo gedit 51-android-rules

For each vendor, append a new line to the file. Each of these lines should have the following format:

SUBSYSTEM=="usb", ATTR{idVendor}=="<vendor_id>", MODE="0666", GROUP="plugdev"

Save the permissions file and quit the editor.

On Mac, no special drivers or permissions are required.

Plug the Android device into your computer's USB port. In Eclipse, select one of the OpenCV sample projects in Package Explorer. Then, from the menu system, select Run | Run as… | Android Application.

An Android Device Chooser window should appear. Your Android device should be listed under Choose a running Android device. (If the device is not listed, try unplugging it and plugging it back in. If that does not work, also try disabling and re-enabling the device's USB debugging option, as described earlier.)

Select the device and click on OK.

If the Auto Monitor Logcat window appears, select the Yes radio button and the verbose drop-down option, and click on OK. This option ensures that all the log output from the application will be visible in Eclipse.

On the Android device you might get a message: OpenCV library package was not found! Try to install it? Make sure the device is connected to the Internet and then touch the Yes button on your device. The Play Store will open to show an OpenCV package. Install the package and then press the hardware back button to return to the sample application, which should be ready for use.

For OpenCV 2.4.3.2, the samples and tutorials have the following functionality:

Sample – 15 puzzle: It splits up a camera feed to make a sliding-block puzzle. The user may swipe blocks to move them.
Sample – color-blob-detection: It detects color regions in a camera feed. The user may touch anywhere to see the outline of a color region.
Sample – face-detection: It draws green rectangles around faces in a camera feed.
Sample – image-manipulations: It applies filters to a camera feed. The user may press the Android menu button to select from a list of filters.
Tutorial 1 – Add OpenCV: It displays a camera feed. The user may press the … menu to select a different camera feed implementation (Java or native C++).
Tutorial 2 – Use OpenCV Camera: It applies filters to a camera feed. The user may press the … menu to select from a list of filters.
Tutorial 3 – Add Native OpenCV: It draws red circles around interest points or features in a camera feed. Generally speaking, interest points or features lie along the high-contrast edges in an image. They are potentially useful in image recognition and tracking applications.
Tutorial 4 – Mix Java+Native OpenCV: It combines the functionality of Tutorial 2 and Tutorial 3. The user may press the … menu to select from a list of filters and a preview of interest points.
Tutorial 5 – Camera Control: It applies filters to a camera feed, which has a customizable resolution. The user may press the … menu to select from a list of filters and a list of resolutions.

Try these applications on your Android device! While an application is running, its log output should appear in the LogCat tab in Eclipse.

Feel free to browse the projects' source code via Package Explorer, to see how they were made. Alternatively, you might want to return to the official samples and tutorials later, after we have built our own project over the course of this book.

Filter reviews by

All

Amazon verified reviews

Amazon Customer Jul 11, 2020

Just started this book...only a couple chapters in. This makes a great book after a couple introductoy books on Java for Android development. I love the direct approach of pushing the compiled code to the phone rather than running it on a virtual device. I will definitely be studying this book...it is what I needed.

Amazon Verified review

Nathan Waters Jun 01, 2016

Short and sweet! Great way to get your feet wet in Computer Vision and Graphics on Android.

HS Aug 26, 2015

GREAT book for new to android programmers. It's light, easy to follow, and provides a good overview of various image processing features. Sample projects are fun and easy to develop or modify for your own use.

Pradeep Jul 26, 2014

Nice and compact book - gets started on OpenCV and Image processing with Android! However, could have been little more elaborate on OpenCV fundamentals too.

Daniel Noonan Mar 18, 2014

This book is tiny. It's incredibly small. It's width is about half that of the diameter of a United States dime. So it's a little under a centimeter thick. That's the first thing I noticed when the book finally arrived. When I previewed the book here on Amazon I thought it clipped the chapter titles. There really are only five chapters. It's an incredibly short programming book compared to others at just 112 pages. The first 30 pages are setting up OpenCv to work with eclipse. So content wise it doesn't even get into programming anything until one is almost a third or a little over a fourth of the way through the book.I feel a little ripped off to be honest. I don't think the book is worth the cost. However it's the only book on the market which covers this topic. What little content there is I guess gets covered well enough. However there's not much. One might be better off doing the tutorials on the OpenCV website and then finding supplementary material online. I think the publishers rushed this one out to market figuring there was a market for it. I wish the author and the publishers would have done more than meet the bare minimum of expectations though.I was really excited to receive this book. It's such an interesting topic. Perhaps I'm a little biased as I did have such high expectations. It's worth reading through to learn some of the material I just don't think it's worth the full price of the book because it's rather skimpy on material. What material there is however is covered well enough as I mentioned.

Android Application Programming with OpenCV: For Java developers OpenCV is a fantastic opportunity to benefit from the popularity of image related mobile apps on Android. This book teaches you all you need to know about computer vision with practical projects.

What do you get with Print?