Book Image

Kinect in Motion - Audio and Visual Tracking by Example

Book Image

Kinect in Motion - Audio and Visual Tracking by Example

Overview of this book

Kinect is a motion-sensing input device by Microsoft for the Xbox 360 video game console and Windows PCs. It provides capabilities to enhance human-machine interaction along with a zero-to-hero journey to engage the user in a multimodal interface dialog with your software solution. Kinect in Motion - Audio and Visual Tracking by Example guides you in developing more than five models you can use to capture gestures, movements, and voice spoken commands. The examples and the theory discussed provide you with the knowledge to let the user become a part of your application. Kinect in Motion - Audio and Visual Tracking by Example is a compact reference on how to master color, depth, skeleton, and audio data streams handled by Kinect for Windows.Starting with an introduction to Kinect and its characteristics, you will first be shown how to master the color data stream with no more than one page of lines of code. Learn how to manage the depth information and map them against the color ones. You will then learn how to define and manage gestures that enable the user to instruct the application simply by moving arms or any other type of natural action. Finally you will complete your journey through a multimodal interface, combining gestures with audio.The book will lead you through many detailed, real-world examples, and even guide you on how to test your application.
Table of Contents (12 chapters)

Summary


In this chapter we introduced Kinect, looking at the key architectural aspects such as the hardware composition and the SDK 1.6 software components. We walked through the color sensor, IR depth sensors, IR emitter, microphone arrays, the tilt motor for changing the Kinect camera angles, and the three-axis accelerometer.

Kinect generates two video streams using the color camera data and the depth information using the depth sensor. Kinect can detect up to six users in its view field and produce a detailed skeleton for two of them. All these characteristics make Kinect an awesome tool for video tracking motion. The Kinect's audio tracking makes the device a remarkable interface for voice recognition. Combining video and audio, Kinect and its SDK 1.6 are an outstanding technology for NUI.

Kinect is not just technology, it is indeed a means of how we can elevate the way users interact with complex software applications and systems. It is a break-through on how we can include NUIs and multimodal interface.

Kinect discloses unlimited opportunities to developers and software architects to design and create modern applications for different industries and lines of business.

The following examples are not meant to be an exhaustive list, but just a starting point that can inspire your creativity and increase your appetite for this technology.

  • Healthcare: This improves the physical rehabilitation process by constantly capturing data of the motion and posture of patient. We can enhance this scenario by allowing doctors to check the patient data remotely streamed by the Kinect sensor.

  • Education/Professional development: This helps in creating safe and more engaging environments based on gamification where students, teachers, and professionals can exercise activities and knowledge. The level of engagement can be increased even further using augmented reality.

  • Retail: This engages customers across multiple channels using the Kinect's multimodal interface. Kinect can be used as a navigation system for virtual windows while shopping online and/or visiting infotainment kiosks.

  • Home automation: This is also known as domotics where, thanks to the Kinect audio and video tracking, we can interact with all the electrical devices installed at our home (lights, washing machine, and so on).

In the next chapter, we will start to develop with the Kinect SDK, utilizing the depth and RGB camera streams. The applied examples will enable our application to optimize the way we manage and tune the streams themselves.