In order to build a good object tracker, we need to understand what characteristics can be used to make our tracking robust and accurate. So, let's take a baby step in that direction and see whether we can use colorspace information to come up with a good visual tracker. One thing to keep in mind is that color information is sensitive to lighting conditions. In real-world applications, you will have to do some preprocessing to take care of that. But for now, let's assume that somebody else is doing that and we are getting clean color images.
There are many different colorspaces, and picking a good one will depend on the different applications that a user is using. While RGB is the native representation on a computer screen, it's not necessarily ideal for humans. When it comes to humans, we give names to colors more naturally...