Book Image

Android Application Programming with OpenCV

By : Joseph Howse
Book Image

Android Application Programming with OpenCV

By: Joseph Howse

Overview of this book

Take a smartphone from your pocket, and within a few seconds, you can snap a photo, manipulate it, and share it with the world. You have just achieved mass production of image data. With a computer vision library such as OpenCV, you can analyze and transform copious amounts of image data in real time on a mobile device. The upshot to this is that you, as developers, can provide mobile users with many new kinds of images, constantly highlighting certain visual features that are of artistic or practical interest. Android is a convenient platform for such experiments because it uses a high-level language (Java), it provides standardized interfaces for sharing image data between applications, and it is mostly open source, so everyone can study its implementation. Android Application Programming with OpenCV is a practical, hands-on guide that covers the fundamental tasks of computer vision—capturing, filtering, and analyzing images-with step-by-step instructions for writing both an application and reusable library classes. Android Application Programming with OpenCV looks at OpenCV's Java bindings for Android and dispels mysteries such as which version of these bindings to use, how to integrate with standard Android functionality for layout, event handling, and data sharing, and how to integrate with OpenGL for rendering. By following the clear, concise, and modular examples provided in this book, you will develop an application that previews, captures, and shares photos with special effects based on color manipulation, edge detection, image tracking, and 3D rendering.Beneath the application layer, you will develop a small but extensible library that you can reuse in your future projects. This library will include filters for selectively modifying an image based on edge detection, 2D and 3D image trackers, and adapters to convert the Android system's camera specifications into OpenCV and OpenGL projection matrices. If you want a quick start in computer vision for Android, then this is the book for you. By the end of Android Application Programming with OpenCV, you will have developed a computer vision application that integrates OpenCV, Android SDK, and OpenGL.
Table of Contents (12 chapters)
Android Application Programming with OpenCV
Credits
About the Author
About the Reviewers
www.PacktPub.com
Preface
Index

Setting up a development environment


Basically, we have a choice among the following approaches:

  1. Install a prepackaged, preconfigured development environment that contains all the components we need.

  2. Install various components separately and configure them to work together. Within this approach, we may do either of the following:

    • Use a prepackaged, preconfigured version of OpenCV

    • Configure and build OpenCV from source

Let's look at each of these alternatives in detail.

Getting a ready-made development environment – Tegra Android Development Pack (TAPD)

Tegra Android Development Pack (TADP) contains a complete, preconfigured development environment for Android, OpenCV, and some other libraries. TADP builds apps that are optimized for NVIDIA's Tegra processors. Despite being optimized for Tegra, the apps are compatible with other hardware too.

Tip

If you are setting up an Android development environment from scratch, I recommend TADP. It contains recent versions of all our required software and its setup process is simple.

TADP also contains some extras that we do not require for this book. For a complete list of TADP's contents, see the official description at https://developer.nvidia.com/tegra-android-development-pack.

To set up TADP, we just need to download and install it from a secure section of NVIDIA's website. Here are the required steps:

  1. Join the NVIDIA Registered Developer Program at https://developer.nvidia.com/user/register. (It is free.)

  2. Log in at https://developer.nvidia.com/user/login.

  3. Complete your user profile at https://developer.nvidia.com/user/me/profile/rdp_profile.

  4. Apply to join the Tegra Registered Developer Program at https://developer.nvidia.com/rdp/applications/tegra-registered-developer-program. (It is free, too!) Wait for NVIDIA to send you an acceptance email. Normally, you might receive it a few minutes after applying.

  5. Go to https://developer.nvidia.com/tegra-resources and find the download link for TADP's latest version. At the time of writing, the latest version is 2.0r2. There are installers for Windows (32-bit or 64-bit), Mac, and Ubuntu (32-bit or 64-bit). Download and run the appropriate installer.

  6. When the installer presents the Installation Directory step, we can enter any destination, which we will refer to as <tadp>. By default, <tadp> is C:\NVPACK (Windows) or ~/NVPACK (Mac and Ubuntu).

  7. When the installer presents the Installation Options step, we may select any option: Complete, Express, or Custom. Compared to an Express installation, a Complete or Custom installation may include additional versions of Android SDK and binary images of Tegra Android OS, which is NVIDIA's customization of Android. If in doubt, choose Express.

  8. When the installer presents the Proxy Configuration step, we may leave all fields blank unless we are using a proxy server.

  9. After finishing all of the installer's configuration steps, wait for TADP's content to be downloaded and installed.

That's all! Before proceeding, let's just take a note of the locations where TADP has installed certain components. For TADP 2.0r2 (the latest version at the time of writing), the locations are as follows:

  • Android SDK is located at <tadp>/android-sdk-macosx. We will refer to this location as <android_sdk>.

  • Android NDK is located at <tadp>/android-ndk-r8d. We will refer to this location as <android_ndk>.

  • OpenCV4Android is located at <tadp>/OpenCV-2.4.3.2-android-sdk-tadp. We will refer to this location as <opencv>.

  • Eclipse is located at <tadp>/eclipse. We will refer to this location as <eclipse>.

Note

The TADP installer automatically edits the system's PATH to include <android_sdk>/platform-tools and <android_sdk>/tools. Also, it creates an environment variable called NDKROOT, whose value is <android_ndk>.

Now, we can proceed to Building the OpenCV Samples with Eclipse, later in this chapter.

Tip

Downloading the example code

You can download the example code files for all Packt books you have purchased from your account at http://www.packtpub.com. If you purchased this book elsewhere, you can visit http://www.packtpub.com/support and register to have the files e-mailed directly to you. The example code for this book is also available from the author's website at http://nummist.com/opencv/.

Assembling a development environment piece-by-piece

Instead of using TADP as a ready-made solution, we may assemble our own development environment. Broadly, this task has two stages:

  • Set up a general-purpose Android development environment

  • Set up OpenCV for use in this environment

Let's start by looking at the setup steps for a general-purpose Android development environment. We will not delve into very much detail here, because good instructions are available at the given links and, because you, as an Android developer, have probably been through similar steps before.

Tip

If you already have an Android development environment and you just want to add components to it, some of the following steps will not apply to you.

Here are the steps:

  1. If we are using Windows or Linux, we may need to obtain JDK 6 manually. (On Mac, if JDK 6 is not present, the operating system will automatically offer to install it when needed.) The JDK 6 installers or packages are available for Windows and many Linux distributions at http://www.oracle.com/technetwork/java/javase/downloads/jdk6downloads-1902814.html. Alternatively, on Linux, check your repository for the JDK packages. Install JDK 6.

  2. Download Eclipse and unzip it to any destination, which we will refer to as <eclipse>. There are many versions from which we may choose. Google provides an Eclipse distribution called Android Developer Tools (ADT) Bundle, which comes with Android SDK and the ADT plugin prepackaged and preconfigured. ADT Bundle is available at http://developer.android.com/sdk/index.html. Many other up-to-date Eclipse distributions are available at http://www.eclipse.org/downloads/. Of these, Eclipse for Mobile Developers is a good and minimalist choice as a foundation for an Android development environment.

  3. If we did not get the ADT Bundle, we now need to set up Android SDK and the ADT plugin for Eclipse. Go to http://developer.android.com/sdk/index.html and get the download named Android SDK Tools. Install or unzip it to any destination, which we will refer to as <android_sdk>. Open Eclipse and install the ADT plugin according to the official instructions at http://developer.android.com/sdk/installing/installing-adt.html. Restart Eclipse. A window, Welcome to Android Development, should appear. Click on Use Existing SDKs, browse to <android_sdk>, and click on Next. Close Eclipse.

  4. If we are using Windows, download and install Cygwin from http://cygwin.com/install.html.

  5. Download Android NDK from http://developer.android.com/tools/sdk/ndk/index.html. Unzip it to any destination, which we will refer to as <android_ndk>.

  6. Edit your system's PATH to include <android_sdk>/platform-tools and <android_sdk>/tools. Also, create an environment variable named NDKROOT with the value as <android_ndk>. (If you are unsure how to edit PATH and other environment variables, see Appendix A: Editing environment variables.)

    Tip

    Editing environment variables on Windows

    The system's Path variable and other environment variables can be edited in the Environment Variables window of Control Panel.

    On Windows Vista/7/8, open the Start menu and launch Control Panel. Now, go to System and Security | System | Advanced system settings. Click on the Environment Variables button.

    On Windows XP, open the Start menu and go to Control Panel | System. Click on the Advanced tab. Click on the Environment Variables button.

    Now, under System variables, select an existing environment variable, such as Path, and click on the Edit button. Alternatively, make a new environment variable by clicking on the New button. Edit the variable's name and value as needed. For example, if we want to add C:\android-sdk\platform-tools and C:\android-sdk\tools to Path, we should append ;C:\android-sdk\platform-tools;C:\android-sdk\tools to the existing value of Path. Note the use of semicolons as separators.

    To apply the changes, click on all the OK buttons until we are back in the main window of Control Panel. Now, log out and again log in.

    Editing environment variables on Mac

    Edit ~/.profile.

    To append to an existing environment variable, add a line such as export PATH=$PATH:~/android-sdk/platform-tools:~/android-sdk/tools. This example appends ~/android-sdk/platform-tools and ~/android-sdk/tools to PATH. Note the use of colons as separators.

    To create a new environment variable, add a line such as export NDKROOT=~/android-ndk.

    Save your changes, log out, and again log in.

    Editing environment variables on Ubuntu

    Edit ~/.pam_environment.

    To append to an existing environment variable, add a line such as PATH DEFAULT=${PATH}:~/android-sdk/platform-tools:~/android-sdk/tools. This example appends ~/android-sdk/platform-tools and ~/android-sdk/tools to PATH. Note the use of colons as separators.

    To create a new environment variable, add a line such as NDKROOT DEFAULT=~/android-ndk.

    Save your changes, log out, and again log in.

Now, we have an Android development environment but we still need OpenCV. We may choose to download a prebuilt version of OpenCV or we may build it from source. These options are discussed in the following two subsections.

Tip

Generally, Android applications should use a prebuilt version of OpenCV. One important reason is that the prebuilt versions are available for Android users as shared libraries, which save disk space and simplify updates.

For the purpose of this book's project, there is no need to build OpenCV from source. We just mention this option for completeness, since it may be of interest to advanced users who want to modify OpenCV.

Getting the prebuilt OpenCV4Android

The prebuilt versions of OpenCV4Android can be downloaded from http://sourceforge.net/projects/opencvlibrary/files/opencv-android/. Look for files that have opencv-android in the name, such as OpenCV-2.4.5-android-sdk.zip (the latest version at the time of writing). Download the latest version and unzip it to any destination, which we will refer to as <opencv>.

Building OpenCV4Android from source

Alternatively, the process for building OpenCV4Android from trunk (the latest, unstable source code) is documented at http://code.opencv.org/projects/opencv/wiki/Building_OpenCV4Android_from_trunk. For a summary of the process for building from trunk, continue reading this section. Otherwise, skip ahead to Building the OpenCV samples with Eclipse, later in this chapter.

Tip

Since trunk contains the latest and unstable source code, there is no guarantee that the build process will succeed. You may need to do your own troubleshooting if you want to build from trunk.

To build OpenCV from source, we need the following additional software:

  • Git: It is a Source Control Management (SCM) tool, which we will use to obtain OpenCV's source code. On Windows or Mac, download and install Git from http://git-scm.com/. On Linux, install it using your package manager. For example, on Ubuntu, open Terminal and run $ sudo apt-get install git-core.

  • CMake: It is a set of build tools. On Windows or Mac, download and install CMake from http://www.cmake.org/cmake/resources/software.html. On Linux, install it using your package manager. For example, on Ubuntu, open Terminal and run $ sudo apt-get install cmake.

  • Apache Ant 1.8.0 or greater: It is a set of build tools for Java. On Linux, just install Ant using your package manager. For example, on Ubuntu, open Terminal and run $ sudo apt-get install ant. On Windows or Mac, download Ant from http://ant.apache.org/bindownload.cgi and unzip it to any destination, which we will refer to as <ant>. Make the following changes to your environment variables:

    • Add <ant>/bin to PATH.

    • Create a variable, ANT_HOME, with the value <ant>.

  • Python 2.6 or greater (but not 3.0 or greater): It is a scripting language that is used by some of the OpenCV build scripts. An appropriate version of Python comes preinstalled on Mac and most Linux systems, including Ubuntu. On Windows, download and install Python from http://www.python.org/getit/. If you have installed multiple versions of Python on your system, ensure that an installation of Python 2.6 or greater (but not 3.0 or greater) is the only one in Path (Windows) or PATH (Mac, Linux, or other Unix-like systems). The OpenCV build scripts do not run properly with Python 3.0 or greater.

Once we have these prerequisites, we may download the OpenCV source code to any location, which we will refer to as <opencv_source>. Then, we may build it using an included script. The steps are platform-specific, and are described as follows:

On Windows, copy <opencv>\android\scripts\wincfg.cmd.tmpl to <opencv>\android\scripts\wincfg.cmd. Edit <opencv>\android\scripts\wincfg.cmd. The locations of several of the prerequisites are declared in this file. Modify them so that they are correct for your system. Save your changes. Then, open Git Bash (Git's command prompt) and run the following commands:

$ git clone git://code.opencv.org/opencv.git <opencv_source>
$ cd <opencv_source>/android
$ scripts/cmake_android.cmd
$ cd build
$ make -j8

On Mac, Ubuntu, or other Unix-like systems, open Terminal (or another command line shell) and run the following commands:

$ git clone git://code.opencv.org/opencv.git <opencv_source>
$ cd <opencv_source>/android
$ sh ./scripts/cmake_android.sh
$ cd build
$ make -j8

If all goes well, we should get a build of OpenCV4Android in <opencv_source>/android/build. We may move it elsewhere if we wish. We will refer to its final location as <opencv>.