This chapter shows you how to use data from a depth camera to identify foreground and background regions, so that we can limit an effect to only the foreground or only the background. As prerequisites, we need a depth camera, such as Microsoft Kinect, and we need to build OpenCV with support for our depth camera. For build instructions, see Chapter 1, Setting Up OpenCV.

We'll deal with two main topics in this chapter: depth estimation and segmentation. We will explore depth estimation with two distinct approaches: firstly, by using a depth camera (a prerequisite of the first part of the chapter), such as Microsoft Kinect, and then, by using stereo images, for which a normal camera will suffice. For instructions on how to build OpenCV with support for depth cameras, see Chapter 1, Setting Up OpenCV. The second part of the chapter is about segmentation, the technique that allows us to extract foreground objects from an image.