Book Image

Augmented Reality with Kinect

By : Rui Wang
Book Image

Augmented Reality with Kinect

By: Rui Wang

Overview of this book

Microsoft Kinect changes the notion of user interface design. It differs from most other user input controllers as it enables users to interact with the program without touching the mouse or a trackpad. It utilizes motion sensing technology and all it needs is a real-time cameras, tracked skeletons, and gestures. Augmented Reality with Kinect will help you get into the world of Microsoft Kinect programming with the C/C++ language. The book will cover the installation, image streaming, skeleton and face tracking, multi-touch cursors and gesture emulation. Finally, you will end up with a complete Kinect-based game. Augmented Reality with Kinect will help you get into the world of Kinect programming, with a few interesting recipes and a relatively complete example. The book will introduce the following topics: the installation and initialization of Kinect applications; capturing color and depth images; obtaining skeleton and face tracking data; emulating multi-touch cursors and gestures; and developing a complete game using Kinect features. The book is divided in such a way so as to ensure that each topic is given the right amount of focus. Beginners will start from the first chapter and build up to developing their own applications.

Related Content you might be interested in

Current Title:

Small book image
Augmented Reality with Kinect
Rui Wang
Jul, 2013 | 122 pages
No titles found
Table of Contents (14 chapters)
Augmented Reality with Kinect
Augmented Reality with Kinect
Credits
Credits
About the Author
About the Author
About the Reviewers
About the Reviewers
www.PacktPub.com
www.PacktPub.com
Preface
Preface
Free Chapter
1
Getting Started with Kinect
Getting Started with Kinect
Installation of Kinect
The idea of the AR-based Fruit Ninja game
Summary
2
Creating Your First Program
Creating Your First Program
Preparing the development environment
Starting the device
Summary
3
Rendering the Player
Rendering the Player
Choosing image stream types
Obtaining color and depth images
An incorrect way to combine depth and color
Aligning color with depth
Using a green screen with Kinect
Summary
4
Skeletal Motion and Face Tracking
Skeletal Motion and Face Tracking
Understanding skeletal mapping
Obtaining joint positions
Drawing the linetrails following the hands
Face tracking in Kinect
Detecting a face from the camera
Constructing the face model
Summary
5
Designing a Touchable User Interface
Designing a Touchable User Interface
Multitouch systems
Locating the cursors
Common touching gestures
Recognizing holding and swiping gestures
Sending cursors to external applications
Summary
6
Implementing the Scene and Gameplay
Implementing the Scene and Gameplay
Integrating the current code
Cutting the fruits
Playing the game
Summary
Where to Go from Here
Where to Go from Here
libfreenect – the pioneer of Kinect middleware
OpenNI – a complete and powerful Kinect middleware
Free and open source resources
Commercial products using Kinect
Index
Index

OpenNI – a complete and powerful Kinect middleware

The OpenNI (Open Natural Interaction) organization was also created in November 2010. It focuses on natural and organic user interfaces and develops its own framework for Kinect devices and uses. In December 2010, PrimeSense, one of the OpenNI members, released its open source drivers and motion-tracking middleware called NITE for Microsoft Kinect.

The OpenNI framework provides a series of APIs fulfilling natural interaction requirements, such as voice recognition, motion tracking, and hand and body gestures. To directly install OpenNI packages and Kinect drivers, we can go to the following web page:

http://www.openni.org/openni-sdk/

Before you download and use OpenNI, you'd better remove all existing Microsoft Kinect drivers from your device manager. These are shown in the following screenshot:

Find and remove Microsoft Kinect drivers from the Windows driver manager

You may also refer to the following article about how to install different drivers:

http://www.codeproject.com/Articles/148251/How-to-Successfully-Install-Kinect-on-Windows-Open

Download appropriate OpenNI Binaries files (stable or unstable). At present, OpenNI provides Windows, Linux, and Mac OS X versions of their SDKs for use.

Install OpenNI by following the given instructions during installation. Please note that the stable version of OpenNI only supports Windows and Ubuntu at present, but its unstable version can even support Mac OS X and ARM platforms.

You may also require the NITE middleware for full-body tracking, accurate user skeleton joint tracking, and gesture recognition. It can be found at:

http://www.openni.org/files/nite/

You will find that NITE currently doesn't support ARM platforms. This means that skeleton tracking will be disabled on embedded platforms.

Now we can start an OpenNI program to see if it works. Run NiViewer.exe from OpenNI\Samples\Bin\Release\. You will see screenshots like the following one:

The result of running NiViewer

You may read something more about OpenNI programming at the following link:

http://www.openni.org/resources/

The OpenNI source code repository can also be found at GitHub:

https://github.com/OpenNI/OpenNI

PrimeSense Sensor Module for OpenNI can be found at:

https://github.com/PrimeSense/Sensor

https://github.com/avin2/SensorKinect

Previous Section
End of Section
Next Section