The optical flow algorithms we have presented so far are very useful, but have limited use in the case of motion compensation in videos shot by a moving camera. In that case, the camera motion must be compensated using other techniques, as the optical flow-based warping does not produce seamless results.
For the camera motion to be estimated, we must focus on one or more points that remain stable, or have known motion and examine the change in their coordinates, which we must then compensate. The points that are chosen must be distinguishable in all frames, so that their appearances can be matched to each other. When we just need to find the correspondence between two small sets of characteristic points in two frames of a video, we are talking about the feature tracking. When the process has to do with finding the correspondences between all points in two frames of the same scene, it is called image registration.
One of the most challenging...