In the previous chapter, we successfully initialized the Kinect device and started the color and depth image streams. Now it is time to read data from the streams at every frame and show them on screen so that we can obtain the very important depth values for many uses. For example, we can change the color image at a specified depth to a different one. Another good idea is to place some virtual objects around the player while rendering the scene. Then we use depths to decide if part of an object is behind or in front of the player to make the compositing result more natural.
In the following sections, we will introduce how to obtain and render color images read by the cameras and depth images read by the depth sensor. We will continue to work on the framework created in the previous chapter, with Kinect already initialized and ready for use.