Optical flow is the pattern of apparent motion between two consecutive frames of video. We will select feature points in the first frame and try to determine where those features have gone in the second frame. This search is subject to a few caveats:
We will make no attempt to distinguish between camera motion and subject motion.
We assume that a feature's color or brightness remains similar between frames.
We assume that neighboring pixels have a similar motion.
OpenCV's calcOpticalFlowPyrLK function implements the Lucas-Kanade method of computing optical flow. Lucas-Kanade relies on a 3 x 3 neighborhood (that is, 9 pixels) around each feature. Taking each feature's neighborhood from the first frame, we will try to find the best matching neighborhood in the second frame, based on the least squared error. OpenCV's implementation of Lucas-Kanade uses an image pyramid, meaning it performs the search at various scales. Thus, it supports large or small motions alike. PyrLk...