On Windows, we have the option of setting up a 32-bit development environment (to make apps that are compatible with both 32-bit and 64-bit Windows) or a 64-bit development environment (to make optimized apps that are compatible with 64-bit Windows only). Recent versions of OpenCV are available in 32-bit and 64-bit versions.

We also have a choice of either using binary installers or compiling OpenCV from source. For our Windows apps in this book, the binary installers provide everything we need. However, we will also discuss the option of compiling from source because it enables us to configure additional features, such as support for Kinect and Asus depth cameras, which might be relevant to your future work or to our projects in other books.

Regardless of our approach to obtain OpenCV, we need a general-purpose C++ development environment and a general-purpose Python 2.7 development environment. We will set up these environments using binary installers.

As our C++ development environment, we will use Visual Studio 2010 or a later version. Use any installation media you might have purchased, or go to the downloads page at http://www.visualstudio.com/en-us/downloads/download-visual-studio-vs.aspx. Download and run the installer for one of the following:

If the installer lists optional C++ components, we should opt to install them all. After the installer runs till completion, reboot.

Installers for Python 2.7 are available at http://www.python.org/getit/. Download and run the latest revision of Python 2.7 in either the 32-bit variant or the 64-bit variant.

To make Python scripts run using our new Python 2.7 installation by default, let's edit the system's Path variable and append ;C:\Python2.7 (assuming Python 2.7 is installed in the default location). Remove any previous Python paths, such as ;C:\Python2.6. Log out and log back in (or reboot).

Let's assume that we also want to use binary installers for NumPy, SciPy, and wxPython. Download and run the installers for the latest stable library versions that target Python 2.7. We can find these installers at the following locations:

Requests does not have a binary installer but we can download the latest source bundle from https://github.com/kennethreitz/requests/archive/master.zip. Unzip it to any destination, which we will refer to as <unzip_destination>. Open Command Prompt and run the following commands:

> cd <unzip_destination>
> python setup.py install

Next, we can put PyInstaller in any convenient location, since it is treated as a set of tools rather than a library. Let's download the latest release version from http://www.pyinstaller.org/ and unzip it to C:\PyInstaller or any another location of your choice.

Now, we are ready to set up OpenCV and, optionally, other computer vision libraries.

> cmake -D CMAKE_BUILD_TYPE=RELEASE -D WITH_OPENNI=ON -D OPENNI_LIB_DIR="<openni_install_destination>\Lib" -D OPENNI_INCLUDE_DIR="<openni_install_destination>\Include" -D OPENNI_PRIME_SENSOR_MODULE_BIN_DIR="<sensorkinect_install_destination>\Bin" -D WITH_TBB=ON -D TBB_LIB_DIR="<tbb_unzip_destination>\lib\ia32\vc10" -D TBB_INCLUDE_DIR="<tbb_unzip_destination>\include" -G "Visual Studio 10" "<opencv_unzip_destination>\opencv\sources"

> cmake -D CMAKE_BUILD_TYPE=RELEASE -D WITH_OPENNI=ON -D OPENNI_LIB_DIR="<openni_install_destination>\Lib" -D OPENNI_INCLUDE_DIR="<openni_install_destination>\Include" -D OPENNI_PRIME_SENSOR_MODULE_BIN_DIR="<sensorkinect_install_destination>\Bin" -D WITH_TBB=ON -D TBB_LIB_DIR="<tbb_unzip_destination>\lib\intel64\vc10" -D TBB_INCLUDE_DIR="<tbb_unzip_destination>\include" -G "Visual Studio 10 Win64" "<opencv_unzip_destination>\opencv\sources"

Let's begin by setting up Xcode and the Xcode Command Line Tools, which give us a complete C++ development environment:

Next, we need a Python 2.7 development environment. Recent versions of Mac come with Python 2.7 preinstalled. However, the preinstalled Python is customized by Apple for the system's internal needs. Normally, we should not install any libraries atop Apple's Python. If we do, our libraries might break during system updates or worse, might conflict with preinstalled libraries that the system requires. Instead, we should install standard Python 2.7 and then install our libraries atop it.

For Mac, there are several possible approaches to obtain standard Python 2.7 and Python-compatible libraries such as OpenCV. All approaches ultimately require OpenCV to be compiled from source using Xcode Command Line Tools. However, depending on the approach, this task is automated for us by third-party tools in various ways. We will look at approaches using MacPorts or Homebrew. These two tools are package managers, which help us resolve dependencies and separate our development libraries from the system libraries.

Regardless of the approach to set up our Python environment, we can put PyInstaller in any convenient location, since it is treated as a set of tools rather than a library. Let's download the latest release version from http://www.pyinstaller.org/ and unzip it to ~/PyInstaller or another location of your choice.

Now, depending on your preference, let's proceed to either the Mac with MacPorts section or the Mac with Homebrew section.

On Raspbian, Ubuntu 13.04 and its later versions, and Ubuntu derivatives such as Linux Mint 15 and its later versions, a recent version of OpenCV is available in the standard repository. This OpenCV package includes support for many video codes and for Intel TBB multiprocessing, but it does not include support for depth cameras (as used in my other book, OpenCV Computer Vision with Python). To install OpenCV and SciPy, open Terminal and run the following commands:

$ sudo apt-get install python-opencv
$ sudo apt-get install python-scipy

On other systems, the standard repository contains OpenCV 2.3.1 but this version is old (from 2011) and lacks some of the functionality used in this book. Thus, we want to compile a newer version of OpenCV from source. Compiling from source is likewise a requirement if we want support for Asus and Kinect depth cameras. Because the dependencies are complex, I have written a script that downloads, configures, and builds OpenCV and related libraries. Here are the steps to obtain and use the script, and then install a few additional elements of our Python environment:

If the script ran successfully, we now have a recent version of OpenCV configured to support C++, Python 2.7, and (on compatible systems) several extras such as video codecs, support for Intel TBB multiprocessing, and support for Kinect and Asus depth cameras. Not all the extras are relevant to our current projects but they might be useful in your future work.

Now we have everything we need to develop OpenCV applications for Debian Wheezy or a derivative. To also develop the same for Android, we need to set up TADP as described in the section Tegra Android Development Pack, later in this chapter.

Recent versions of OpenCV, SciPy, Requests, and wxPython are in the standard repository. To install them, open Terminal and run the following commands:

$ sudo yum install opencv-python
$ sudo yum install scipy
$ sudo yum install python-requests
$ sudo yum install wxPython

Download the latest release version of PyInstaller from http://www.pyinstaller.org/ and unzip it to ~/PyInstaller or another location of your choice.

As an alternative to using the packaged build of OpenCV, the following official tutorial provides instructions to install OpenCV's dependencies and compile it from source: http://docs.opencv.org/trunk/doc/py_tutorials/py_setup/py_setup_in_fedora/py_setup_in_fedora.html.

For optional dependencies and additional compilation options, including depth camera support, refer to my Debian-compatible build script as a starting point: http://nummist.com/opencv/install_opencv_debian_wheezy.sh.

Now we have everything we need to develop OpenCV applications for Fedora or a derivative. To also develop the same for Android, we need to set up TADP as described in the section Tegra Android Development Pack, later in the chapter.

Recent versions of OpenCV, SciPy, Requests, and wxPython are in the standard repository. To install them, open Terminal and run the following commands:

$ sudo yum install python-opencv
$ sudo yum install python-scipy
$ sudo yum install python-requests
$ sudo yum install python-wxWidgets

Download the latest release version of PyInstaller from http://www.pyinstaller.org/ and unzip it to ~/PyInstaller or another location of your choice.

As an alternative to using the packaged build of OpenCV, the steps to install OpenCV from source should be similar to the steps on Fedora, though dependencies' package names might differ.

Next, we need to follow the cross-platform steps to set up an Android development environment.

Tegra Android Development Pack (TADP) contains a complete, preconfigured development environment for Android, OpenCV, and some other libraries. TADP builds apps that are optimized for NVIDIA's Tegra processors. Despite being optimized for Tegra, the apps are compatible with other hardware too.

To set up TADP, we just need to download and install it from a secure section of NVIDIA's website. Here are the required steps:

That's all! Before proceeding, let's just take a note of the locations where TADP has installed certain components. For TADP 3.0r4 (the latest version at the time of writing), the locations are as follows:

The TADP installer automatically edits the system's PATH to include <android_sdk>/platform-tools and <android_sdk>/ tools. Also, it creates an environment variable called NDKROOT, whose value is <android_ndk>.

Building OpenCV Android sample projects with Eclipse

By building and running a few sample applications, we can test our OpenCV installation. At the same time, we can practice using Eclipse.

Let's start by launching Eclipse. The Eclipse launcher should be located at <eclipse>/eclipse.exe (in Windows), <eclipse>/Eclipse.app (in Mac), or <eclipse>/eclipse (in Linux). Run it.

As shown in the following screenshot, we should see a window called Workspace Launcher, which asks us to select a workspace:

A workspace is the root directory for a set of related Eclipse projects. Enter <tadp>/nvsample_workspace, which is a workspace where the OpenCV library, samples, and tutorials are already set up as projects.

Tip

We can return to Workspace Launcher anytime via the menu: File | Switch Workspace | Other….

If the Welcome to Eclipse screen appears, click on the Workbench button.

Now, we should see a window with several panels, including Package Explorer. The OpenCV library, samples, and tutorials should be listed in Package Explorer. We might need to fix some configuration issues in these projects. Our development environment can have different paths and different versions of the Android SDK, than the ones in the sample's default configuration. Any resulting errors are reported in the Problems tab as shown in the following screenshot:

Tip

We should start by resolving any errors in the OpenCV Library project as the samples and tutorials depend on the library.

The following are some of the common configuration problems and their symptoms and solutions:

• The target Android version might not be properly specified. The symptoms of these are that the imports from the java and android packages fail, and there are error messages such as The project was not built since its build path is incomplete. The solution is to right-click on the project in Package Explorer, select Properties from the context menu, select the Android section, and checkmark one of the available Android versions. These steps should be repeated for all projects. At compile time, OpenCV and its samples must target Android 3.0 (API level 11) or greater, though at runtime they also support Android 2.2 (API level 8) or greater.

  

• If we are working on Mac or Linux, the C++ samples might be misconfigured to use the Windows build executable. The symptom of this problem is an error message such as Program "{ndk}/ndk-build.cmd" not found in PATH. The solution is to right-click on the project in Package Explorer, select Properties from the context menu, select the C/C++ Build section, and edit the Build command: field to remove the .cmd extension. These steps should be repeated for all the native (C++) projects, which include OpenCV Sample - face-detection and OpenCV Tutorial 2 - Mixed Processing as shown in the following screenshot:

  

  Tip

  We only need to troubleshoot the projects that have names starting with OpenCV. For this book's purposes, the other TADP samples are not relevant.

Once the OpenCV projects no longer show any errors, we can prepare to test them on an Android device. Remember that the device must have Android 2.2 (Froyo) or a later version, and a camera. For Eclipse to communicate with the device, we must enable the device's USB debugging option with the help of the following steps:

1. Open the Settings app.

2. On Android 4.2 or a later version, go to the About phone or About tablet section and tap Build number seven times. This step enables the Developer options section.

3. Go to the Developer options section (on Android 4.0 or a later version) or the Applications | Development section (on Android 3.2 or an earlier version). Enable the USB debugging option. Now, open the Play Store app and find and install the OpenCV Manager app. OpenCV Manager takes care of checking for any OpenCV library updates when we run any OpenCV applications.

   Tip

   If you do not have the Play Store app on your device, then you need to install OpenCV Manager and certain OpenCV libraries via USB as per the instructions at http://docs.opencv.org/android/service/doc/UseCases.html.

Now, we must prepare our main computer for communication with the Android device. The required steps vary depending on our operating system:

• On Windows, we need to install the proper USB drivers for the Android device. Different vendors and devices have different drivers. The official Android documentation provides links to the various vendors' driver download sites at http://developer.android.com/tools/extras/oem-usb.html#Drivers.

• On Linux, before connecting an Android device via USB, we must specify the device's vendor in a permissions file. Each vendor has a unique ID number, as listed in the official Android documentation at http://developer.android.com/tools/device.html#VendorIds. We will refer to this ID number as <vendor_id>. To create the permissions file, open a command prompt application (such as Terminal) and run the following commands:

  $ cd /etc/udev/rules.d/
  $ sudo touch 51-android.rules
  $ sudo chmod a+r 51-android-rules
  

  Note that the permissions file needs to have root ownership, so we will use sudo while creating or modifying it. Now, open the file in an editor such as gedit:

  $ sudo gedit 51-android-rules
  

  For each vendor, append a new line to the file. Each of these lines should have the following format:

  SUBSYSTEM=="usb", ATTR{idVendor}=="<vendor_id>", MODE="0666",
  GROUP="plugdev"
  

  Save the permissions file and quit the editor.

• On Mac, no special drivers or permissions are required.

Plug the Android device into your computer's USB port. In Eclipse, select one of the OpenCV sample projects in Package Explorer. Then, from the menu system, navigate to Run | Run As... | Android Application:

An Android Device Chooser window should appear. Your Android device should be listed under Choose a running Android device. (If the device is not listed, try unplugging it and plugging it back in. If that does not work, also try disabling and re-enabling the device's USB debugging option, as described earlier.)

Select the device and click on OK.

If the Auto Monitor Logcat window appears, select the Yes radio button and the verbose drop-down option and click on OK. This option ensures that all the log output from the application will be visible in Eclipse.

On the Android device, you might get a message, OpenCV library package was not found! Try to install it?. Make sure that the device is connected to the Internet and then click on the Yes button on your device. The Play Store will open to show an OpenCV package. Install the package and then press the physical back button to return to the sample application, which should be ready for use.

For OpenCV 2.4.8.2, the samples and tutorials have the following functionality:

• Sample – 15 puzzle: This splits up a camera feed to make a sliding block puzzle. The user can swipe blocks to move them.

• Sample – camera-calibration: This estimates the projection and distortion characteristics of the camera. The user prints a test pattern located at https://raw.githubusercontent.com/Itseez/opencv/2.4/doc/acircles_pattern.png and then takes pictures of it from various angles by tapping the screen. After taking several pictures, the user must press the … menu and the Calibrate button to run the calibration algorithm.

• Sample – color-blob-detection: This detects color regions in a camera feed. The user can touch anywhere to see the outline of a color region.

• Sample – face-detection: This draws green rectangles around faces in a camera feed.

• Sample – image-manipulations: This applies filters to a camera feed. The user can press the Android menu button to select from a list of filters.

• Sample – native-activity: This displays a camera feed using native (C++) code.

• Tutorial 1 – Camera Preview: This displays a camera feed. The user can press the … menu to select a different camera feed implementation (Java or native C++).

• Tutorial 2 – Mixed Processing: This applies filters to a camera feed using native (C++) code. The user can press the … menu to select from a list of filters. One of the filters draws red circles around interest points or features in a camera feed. Generally speaking, interest points or features lie along the high-contrast edges in an image. They are potentially useful in image recognition and tracking applications.

• Tutorial 3 – Camera Control: This applies filters to a camera feed, which has a customizable resolution. The user can press the … menu to select from a list of filters and a list of resolutions.

Try these applications on your Android device! While an application is running, its log output should appear in the LogCat tab in Eclipse as shown in the following screenshot:

Feel free to browse the project's source code via Package Explorer to see how it was made. Alternatively, you might want to return to the official samples and tutorials later, after we have built our own projects over the course of this book.

Unity is a 3D game engine that supports development on Windows or Mac and deployment to many platforms, including Windows, Mac, Linux, iOS, Android, a web browser plugin, and several game consoles. For one of our projects, we will use an OpenCV plugin for Unity.

Unity comes in two editions, Standard and Pro, which both support the plugin that we want to use. The Standard edition is free. The Pro edition has a free 30-day trial; otherwise, its price starts from $75 a month or a one-time payment of $1,500. If you do not already have a Unity Pro license, wait until you are ready to start working on our Unity project (which comes late in the book) in case you want to try out some Pro-only functionality at the same time. Once you are ready, download and install the trial from http://unity3d.com/unity/download.

Even before installing Unity, we can get inspiration from the playable demos at https://unity3d.com/gallery/demos/live-demos. Most of these demos are web-based. You will be prompted to download and install the free Unity Web Player when you navigate to one of the web-based games.

After installing Unity, we can learn from other demo projects that include complete source code and art assets. They can be downloaded and opened from https://unity3d.com/gallery/demos/demo-projects. Also check out the tutorials, videos, and documentation at https://unity3d.com/learn.

As you can see, there are a lot of official resources for Unity beginners, so I will let you explore these on your own for now.