Building your first Android project with OpenCV

With the development environment up and running and having the proper NDK background, I can start assembling the big picture on how you can use the OpenCV libraries in your Android application.

OpenCV for Android supports access to its functions through its native API and also its Java wrappers API. In the case of a native API, you will define your native library using Android NDK and include the OpenCV libraries that you are using. Then, you will call your native library from the Java code using Java Native Interface (JNI).

The other option is to use the OpenCV Java wrappers directly in your Java code using the usual Java imports. What will happen is that the Java wrappers will channel your calls to the native OpenCV libraries using JNI.

Of course, it is up to you to choose which style to go with; however, you should understand that using native calls could result in less JNI overhead but require more programming effort. On the other hand, using Java wrappers could cause more JNI overhead with less programming effort.

Note

Consider this scenario: you are processing a video frame or still image and in your algorithm, you will call several OpenCV functions. In this case, it is better to write one native library that calls all these functions. In your Android application, you can access this native library using only one JNI call.

HelloVisionWorld Android application

We will build our first Android application to grab preview frames from the camera in real time and display the preview on a fullscreen using OpenCV's Java camera API.

Creating a project in Eclipse

Following are the steps to create a project in Eclipse:

1. Launch Eclipse and create a new workspace.
2. Create a new Android project and name your application HelloVisionWorld.
3. Set the Minimum SDK version. To build with OpenCV4Android SDK, the minimum SDK version is 11; however, it is highly recommended to use API 15 or higher. In my case, I used API 15.
4. Select Target SDK. In my case, I set it to API 19. Click Next.
5. Allow Eclipse to create New Blank Activity and name it HelloVisionActivity with a layout named activity_hello_vision.
6. Import the OpenCV library project to your workspace. Navigate to File | Import | Existing Android Code Into Workspace.
7. Select the root directory of OpenCV4Android SDK. Deselect All the sample projects and select only OpenCV Library and click Finish:
   
8. Reference the OpenCV library from your Android project. Click Project | Properties. Select the Android tree node from the left-hand side pane and in the right-hand side pane, click Add in the Library section and then OK:
   

Creating a project in Android Studio

Following are the steps to create a project in Android Studio:

1. Launch Android Studio.
2. Create a new Android Studio project and name it HelloVisionWorld with Company Domain set to app0.com.
3. Choose Minimum SDK. To build with OpenCV4Android SDK, the Minimum SDK version is 11.
4. Create a blank activity and name it HelloVisionActivity.
5. To add OpenCV as a dependency to your project, navigate to File | New | Import Module and <OpenCV4Android_Directoy>\sdk\java. Then, click OK. At this point, you may face some issues depending on the components installed from Android SDK. Android Studio will propose quick-fix links to solve such errors and it should be a straightforward fix.
6. Right-click on your newly created application in the project view and choose Open Module Settings or press F4.
7. In the Dependencies tab, press the + button and select Module Dependency.
8. Choose the OpenCV library and press Add. Now, you should be able to import the OpenCV classes to your project.

Moving forward, you should be able to follow the steps regardless of your choice of an IDE:

1. Open the layout file and edit it to match the following code. We added the OpenCV namespace and defined a Java camera view layout:

   <RelativeLayout xmlns:android="http://schemas.android.com/apk/res/android"
       xmlns:tools="http://schemas.android.com/tools"
       xmlns:opencv="http://schemas.android.com/apk/res-auto"
       android:layout_width="match_parent"
       android:layout_height="match_parent"
       android:paddingBottom="@dimen/activity_vertical_margin"
       android:paddingLeft="@dimen/activity_horizontal_margin"
       android:paddingRight="@dimen/activity_horizontal_margin"
       android:paddingTop="@dimen/activity_vertical_margin"
       tools:context="com.example.hellovisionworld.HelloVisionActivity" >
       <org.opencv.android.JavaCameraView
           android:layout_width="fill_parent"
           android:layout_height="fill_parent"
           android:visibility="gone"
           android:id="@+id/HelloVisionView"
           opencv:show_fps="true"
           opencv:camera_id="any" />
   </RelativeLayout>

   Note

   Downloading the example code

   You can download the example code files for all the Packt books that you have purchased from your account at http://www.packtpub.com. If you purchased this book elsewhere, you can visit http://www.packtpub.com/support and register in order to have the files e-mailed to you directly.

2. As we will be using the device camera, we need to set some permissions in the AndroidManifest file:

   </application>
   
   <uses-permission android:name="android.permission.CAMERA"/>
   
   <uses-feature android:name="android.hardware.camera" android:required="false"/>
   <uses-feature android:name="android.hardware.camera.autofocus" android:required="false"/>
   <uses-feature android:name="android.hardware.camera.front" android:required="false"/>
   <uses-feature android:name="android.hardware.camera.front.autofocus" android:required="false"/>

3. Hide the title and system buttons in the AndroidManifest file:

   <application
       android:icon="@drawable/icon"
       android:label="@string/app_name"
       android:theme="@android:style/Theme.NoTitleBar.Fullscreen" >

4. We need to initialize the OpenCV library in the created activity. To do so, we use asynchronous initialization using the OpenCV Manager service to access the OpenCV libraries externally installed in the target system. First, we need to install the OpenCV Manager on the emulator that we will use. To do so, use the adb install command in the command prompt:

   adb install <OpenCV4Android SDK_Home>\apk\OpenCV_2.4.X_Manager_2.X_<platform>.apk
   

   Replace <OpenCV4Android SDK_Home> with your OpenCV installation folder and X in the apk name with the available versions in your apk folder.

   For <platform>, use the following table to choose which platform to install according to the system image that is installed on your emulator:

   Hardware platform	Package name
   armeabi-v7a (ARMv7-A + NEON)	OpenCV_2.4.X_Manager_2.X_armv7a-neon.apk
   armeabi (ARMv5, ARMv6)	OpenCV_2.4.X_Manager_2.X_armeabi.apk
   Intel x86	OpenCV_2.4.X_Manager_2.X_x86.apk
   MIPS	OpenCV_2.4.X_Manager_2.X_mips.apk

   Note

   When you are testing your application on a real device, a message will be displayed asking you to download the OpenCV manager from Google Play, so click Yes and check which version of OpenCV it supports so that you can load it through asynchronous initialization.

5. In Activity, define the following and fix the imports accordingly:

   //A Tag to filter the log messages
   private static final String  TAG = "Example::HelloVisionWorld::Activity";
   
   //A class used to implement the interaction between OpenCV and the //device camera.
   private CameraBridgeViewBase mOpenCvCameraView;
   
   //This is the callback object used when we initialize the OpenCV //library asynchronously
   private BaseLoaderCallback mLoaderCallback = new BaseLoaderCallback(this) {
   
       @Override
          //This is the callback method called once the OpenCV //manager is connected
       public void onManagerConnected(int status) {
         switch (status) {
     //Once the OpenCV manager is successfully connected we can enable the camera interaction with the defined OpenCV camera view
         case LoaderCallbackInterface.SUCCESS:
           {
             Log.i(TAG, "OpenCV loaded successfully");
             mOpenCvCameraView.enableView();
           } break;
             default:
               {
                 super.onManagerConnected(status);
               } break;
          }
       }
   };

6. Update the onResume activity callback method to load the OpenCV library and fix the imports accordingly:

   @Override
   public void onResume(){
   
     super.onResume();
   
   //Call the async initialization and pass the callback object we //created later, and chose which version of OpenCV library to //load. Just make sure that the OpenCV manager you installed //supports the version you are trying to load.
     OpenCVLoader.initAsync(OpenCVLoader.OPENCV_VERSION_2_4_10, this, mLoaderCallback);
   }

7. Your activity needs to implement CvCameraViewListener2 to be able to receive camera frames from the OpenCV camera view:

   public class HelloVisionActivity extends Activity implements CvCameraViewListener2

8. Fix the imports error accordingly and also insert the unimplemented methods in your activity.
9. In the onCreate activity callback method, we need to set the OpenCV camera view as visible and register your activity as the callback object that will handle the camera frames:

   @Override
   protected void onCreate(Bundle savedInstanceState) {
     Log.i(TAG, "called onCreate");
   
     super.onCreate(savedInstanceState);
       getWindow().addFlags(WindowManager.LayoutParams.FLAG_KEEP_SCREEN_ON);
     setContentView(R.layout.activity_hello_vision);
   
     mOpenCvCameraView = (CameraBridgeViewBase) findViewById(R.id.HelloVisionView);
   
     //Set the view as visible
     mOpenCvCameraView.setVisibility(SurfaceView.VISIBLE);
   
     //Register your activity as the callback object to handle //camera frames
     mOpenCvCameraView.setCvCameraViewListener(this);
   }

10. The last step is to receive the camera frames. In order to do so, change the implementation of the onCameraFrame callback method:

    public Mat onCameraFrame(CvCameraViewFrame inputFrame) {
    
      //We're returning the colored frame as is to be rendered on //thescreen.
      return inputFrame.rgba();
    }

11. Now you're ready to build and install your application on the emulator or on a real device.
12. This is the application running on an emulated camera: