Using Kinect's Microphone Array | Kinect for Windows SDK Programming Guide

Sign In Start Free Trial

Book Overview & Buying
Table Of Contents

Kinect for Windows SDK Programming Guide

By : Abhijit Jana

4.6 (16)

Kinect for Windows SDK Programming Guide

4.6 (16)

By: Abhijit Jana

Overview of this book

Build motion-sensing applications with Microsoft\'s Kinect for Windows SDK quickly and easily with this book and ebook.

Kinect for Windows SDK Programming Guide

Kinect for Windows SDK Programming Guide

Credits

Credits

About the Author

About the Author

Acknowledgement

Acknowledgement

About the Reviewers

About the Reviewers

www.PacktPub.com

www.PacktPub.com

Preface

Preface

Free Chapter

Understanding the Kinect Device

Understanding the Kinect Device

Components of Kinect for Windows

Kinect for Windows versus Kinect for Xbox

Where can you use Kinect

Summary

Getting Started

Getting Started

System requirements for the Kinect for Windows SDK

Evaluation of the Kinect for Windows SDK

Downloading the SDK and the Developer Toolkit

Installing Kinect for Windows SDK

Testing your device

Looking inside the Kinect SDK

Features of the Kinect for Windows SDK

The Kinect for Windows Developer Toolkit

Making your development setup ready

The Coding4Fun Kinect Toolkit

Summary

Starting to Build Kinect Applications

Starting to Build Kinect Applications

How applications interact with the Kinect sensor

Kinect Info Box – your first Kinect application

Dealing with the Kinect status

Summary

Getting the Most out of Kinect Camera

Getting the Most out of Kinect Camera

Understanding the Kinect image stream

Different ways of retrieving the color stream from Kinect

KinectCam – a Kinect camera application

Enabling the color stream channel

Looking inside color image stream helpers

Capturing frames on demand

Extending the KinectCam

Applying more effects to the camera

Seeing in low light

Making your application perform better

Using the Coding4Fun toolkit

Summary

The Depth Data – Making Things Happen

The Depth Data – Making Things Happen

Understanding the depth data stream

Capturing and processing depth data

Looking inside depth image stream helpers

Depth data and distance

Working with depth range

Depth data distribution

Player index with depth data

Getting the depth and player index automatically

A 3D view of depth data

Summary

Human Skeleton Tracking

Human Skeleton Tracking

How skeleton tracking works

Skeleton tracking with the Kinect SDK

Start tracking skeleton joints

Flow – capturing skeleton data

An intrusion detector camera application

Looking inside skeleton stream helpers

Skeleton-tracking mode

Skeleton tracking in near mode

The Skeleton

Choosing which skeleton to track

The building blocks – Joints and JointCollection

Steps to be followed for joint tracking

Create your own joints data point

Bones – connecting joints

Adjusting the Kinect sensor automatically and giving live feedback to users

Skeleton smoothing – soften the skeleton's movement

Skeleton space transformation

The Advanced Skeleton Viewer application

Debugging the applications

Getting data frames together

Summary

Using Kinect's Microphone Array

Using Kinect's Microphone Array

Verifying the Kinect audio configuration

Using the Kinect microphone array with your computer

The Kinect SDK architecture for Audio

Kinect microphone array

Audio signal processing in Kinect

Taking control over the microphone array

Kinect sound recorder – capturing Kinect audio data

Processing the audio data

Sound source localization

Summary

Speech Recognition

Speech Recognition

How speech recognition works

Using Kinect with your Windows PC speech recognition

Beginning with Microsoft Speech API (SAPI)

Draw What I Want – a speech-enabled application

Summary

Building Gesture-controlled Applications

Building Gesture-controlled Applications

What is a gesture

Approaches for gesture recognition

Basic gesture recognition

Algorithmic gesture recognition

Weighted network gesture recognition

Template-based gesture recognition

Building gesture-enabled controls

The Basic Interaction – a WPF application

Key things to remember

Summary

Developing Applications Using Multiple Kinects

Developing Applications Using Multiple Kinects

Setting up the environment for multiple Kinects

Multiple Kinects – how to reduce interference

Detecting multiple Kinects

Developing an application with multiple Kinects

Controlling multiple sensor status changes

Handling a failover scenario using Kinects

Challenges faced in developing applications using multiple Kinects

Applications where multiple Kinects can be used

Summary

Putting Things Together

Putting Things Together

Taking Kinect to the Cloud

Remotely using the Kinect with Windows Phone

Using Kinect with a Netduino microcontroller

Augmented reality applications

Working with face tracking

Working with XNA and a 3D avatar

Summary

Index

Index

Processing the audio data

The KinectAudioSource class not only helps to capture the audio data from the sensors but also offers to take control over many aspects of audio data processing by interacting with the underlying DMO.

Echo cancellation

Echo cancellation helps to increase the sound quality. The Acoustic Echo Cancellation (AEC) component within the audio-processing pipeline is responsible for removing the echoes that we are sending to the microphone. To make the AEC work, you need to set the EchoCancellationMode property of kinectAudioSource.

EchoCancellationMode uses a set of enumeration values to set the type of echo cancellation as listed in the following table:

Name	Description
`CancellationOnly`	With the value of `CancellationOnly`, the audio DMO will perform only cancellation of acoustic echo.
`CancellationAndSuppression`	This enables the cancellation of echo as we perform the echo suppression using AES (Acoustic Echo Suppression ).
`None`	No echo cancellation will be performed...

CONTINUE READING

83

Tech Concepts

36

Programming languages

73

Tech Tools

Unlimited access to the largest independent learning library in tech of over 8,000 expert-authored tech books and videos.

Innovative learning tools, including AI book assistants, code context explainers, and text-to-speech.

50+ new titles added per month and exclusive early access to books as they are being written.

Kinect for Windows SDK Programming Guide

Search

Your notes and bookmarks