Before we jump into OpenCV functionalities, we need to understand why those functions were built in the first place. It's important to understand how the human visual system works so that you can develop the right algorithms. The goal of the Computer Vision algorithms is to understand the content of images and videos. Humans seem to do it effortlessly! So, how do we get machines to do it with the same accuracy?
Let's consider the following figure:
The human eye captures all the information that comes along such as color, shapes, brightness, and so on. In the preceding image, the human eye captures all the information about the two main objects and stores it in a certain way. Once we understand how our system works, we can take advantage of this to achieve what we want. For example, here are a few things we need to know:
Our visual system is more sensitive to low frequency content than high frequency content. Low frequency content refers to planar regions where pixel values don't change rapidly and high frequency content refers to regions with corners and edges, where pixel values fluctuate a lot. You will have noticed that we can easily see if there are blotches on a planar surface, but it's difficult to spot something like that on a highly textured surface.
The human eye is more sensitive to changes in brightness as compared to changes in color.
Our visual system is sensitive to motion. We can quickly recognize if something is moving in our field of vision even though we are not directly looking at it.
We tend to make a mental note of salient points in our field of vision. Let's consider a white table with four black legs and a red dot at one of the corners of the table surface. When you look at this table, you'll immediately make a mental note that the surface and legs have opposing colors and there is a red dot on one of the corners. Our brain is really smart that way! We do this automatically so that we can immediately recognize it if we encounter it again.
To get an idea of our field of view, let's take a look at the top view of a human and the angles at which we see various things:
Our visual system is actually capable of a lot more things, but this should be good enough to get us started. You can explore further by reading up on Human Visual System Models on the internet.