This book will show you how to use OpenCV's Java bindings in an Android app that displays a camera feed, saves and shares photos, manipulates colors and edges, and tracks real-world objects in 2D or 3D. Integration with OpenGL is also introduced so that you can start building augmented reality (AR) apps that superimpose virtual 3D scenes on tracked objects in the camera feed.
OpenCV is an open-source, cross-platform library that provides building blocks for computer vision experiments and applications. It offers high-level interfaces for capturing, processing, and presenting image data. For example, it abstracts away details about camera hardware and array allocation. OpenCV is widely used in both academia and industry.
Android is a mobile operating system that is mostly open source. For Java developers, it offers a high-level application framework called Android SDK. Android apps are modular insofar as they have standard, high-level interfaces for launching each other and sharing data. Mobility, a high level of abstraction, and data sharing are great starting points for a photo sharing app, similar to the one we will build.
Although OpenCV and Android provide a lot of high-level abstractions (and a lot of open source code for curious users to browse), they are not necessarily easy for newcomers. Setting up an appropriate development environment and translating the libraries' broad functionality into app features are both daunting tasks. This concise book helps by placing an emphasis on clean setup, clean application design, and a simple understanding of each function's purpose.
The need for a book on this subject is particularly great because the OpenCV's Java and Android bindings are quite new and their documentation is not yet mature. Little has been written about the steps for integrating OpenCV with an Android's standard camera, media, and graphics APIs. Surely integration is a major part of an app developer's work, so it is a major focus of this book.
By the end of our journey together, you will have a taste of the breadth of application features that are made possible by integrating OpenCV with other Android libraries. You will have your own small library of reusable classes that you can extend or modify for your future computer vision projects. You will have a development environment and the knowledge to use it, and you will be able to make more apps!
Chapter 1, Setting Up OpenCV, covers the steps to setting up OpenCV and an Android development environment, including Eclipse and Android SDK.
Chapter 2, Working with Camera Frames, shows how to integrate OpenCV into an Android app that can preview, capture, save, and share photos.
Chapter 3, Applying Image Effects, explores the OpenCV's functionality for manipulating color channels and neighborhoods of pixels. We expand our app to include channel-mixing filters, "curve" filters, and a filter that darkens edges.
Chapter 4, Recognizing and Tracking Images, demonstrates the steps to recognizing and tracking a known target (such as a painting) when it appears in a video feed. We expand our app so that it draws an outline around any tracked target.
Chapter 5, Combining Image Tracking with 3D Rendering, improves upon our previous tracking technique by determining the target's position and rotation in real 3D space. We expand our app so that it sets up an OpenGL 3D scene with the same perspective as the Android device's real camera. Then, we draw a 3D cube atop any tracked target.
This book provides setup instructions for OpenCV and an Android development environment, including Eclipse and Android SDK. The software is cross platform and the instructions cover Windows, Mac, and Linux. Other Unix-like environments may work, too, if you are willing to do your own tailoring of the setup steps.
You need a mobile device running Android 2.2 (Froyo) or greater and it must have a camera. Preferably, it should have two cameras, front and rear. Also, it should preferably come with the Google Play Store app because OpenCV uses Google Play Store to manage installation and upgrades of shared libraries.
This book is great for Java developers who are new to computer vision and who like to learn through application development. It is assumed that you have previous experience in Java but not necessarily Android. A basic understanding of image data (for example, pixels, color channels) would be helpful, too.
In this book, you will find a number of styles of text that distinguish between different kinds of information. Here are some examples of these styles, and an explanation of their meaning.
Code words in text are shown as follows: "Edit your system's PATH to include <android_sdk>/platform-tools and <android_sdk>/tools."
A block of code is set as follows:
<?xml version="1.0" encoding="utf-8"?> <manifest xmlns:android= "http://schemas.android.com/apk/res/android" package="com.nummist.secondsight" android:versionCode="1" android:versionName="1.0">
When we wish to draw your attention to a particular part of a code block, the relevant lines or items are set in bold:
mCameraView.enableView();
mBgr = new Mat();
mCurveFilters = new Filter[] {
new NoneFilter(),
new PortraCurveFilter(),
new ProviaCurveFilter(),
Any command-line input or output is written as follows:
$ cd /etc/udev/rules.d/ $ sudo touch 51-android.rules $ sudo chmod a+r 51-android-rules
New terms and important words are shown in bold. Words that you see on the screen, in menus or dialog boxes for example, appear in the text like this: "clicking on the Next button moves you to the next screen".
Feedback from our readers is always welcome. Let us know what you think about this book—what you liked or may have disliked. Reader feedback is important for us to develop titles that you really get the most out of.
To send us general feedback, simply send an e-mail to <[email protected]>, and mention the book title via the subject of your message.
If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, see our author guide on www.packtpub.com/authors.
Now that you are the proud owner of a Packt book, we have a number of things to help you to get the most from your purchase.
You can download the example code files for all Packt books you have purchased from your account at http://www.packtpub.com. If you purchased this book elsewhere, you can visit http://www.packtpub.com/support and register to have the files e-mailed directly to you. The example code for this book is also available from the author's website at http://nummist.com/opencv/.
Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you find a mistake in one of our books—maybe a mistake in the text or the code—we would be grateful if you would report this to us. By doing so, you can save other readers from frustration and help us improve subsequent versions of this book. If you find any errata, please report them by visiting http://www.packtpub.com/submit-errata, selecting your book, clicking on the errata submission form link, and entering the details of your errata. Once your errata are verified, your submission will be accepted and the errata will be uploaded on our website, or added to any list of existing errata, under the Errata section of that title. Any existing errata can be viewed by selecting your title from http://www.packtpub.com/support.
Piracy of copyright material on the Internet is an ongoing problem across all media. At Packt, we take the protection of our copyright and licenses very seriously. If you come across any illegal copies of our works, in any form, on the Internet, please provide us with the location address or website name immediately so that we can pursue a remedy.
Please contact us at <[email protected]> with a link to the suspected pirated material.
We appreciate your help in protecting our authors, and our ability to bring you valuable content.
You can contact us at <[email protected]> if you are having a problem with any aspect of the book, and we will do our best to address it. You can also contact the author directly at <[email protected]> or you can check his website, http://nummist.com/opencv/, for answers to common questions about this book.