Instance segmentation is the last thing we will look at in this chapter. In many ways, it can be thought of as a fusion of object detection and semantic segmentation. However, it is definitely a step up in difficulty compared to those two problems.
With instance segmentation, the idea is to find every occurrence, what is called an instance, of a desired object or objects within an image. Once these are found, we want to segment off each instance from the other, even if they belong to the same class of objects. In other words, labels are both class-aware (such as car, sign, or person) and instance-aware (such as car 1, car 2, or car 3).
The result of instance segmentation will look something like this:
The similarity between this and semantic segmentation is clear; we still label pixels according to what object they belong to. However, while semantic segmentation has no knowledge of how many times a certain object occurs within an image instance, segmentation does.
This...