Having seen local and global motion estimation, we will now take a look at object tracking. Tracking objects is one of the most important applications of computer vision. The Kanade-Lucas-Tomasi (KLT) tracker implements an optical flow to track objects in videos. The steps to implement the algorithm are explained as follows:
Detect Harris corners in the first frame of the video.
For each detected Harris corner, compute the motion between consecutive frames using the optical flow (translator) and local affine transformation (affine).
Now link these motion vectors from frame-to-frame to track the corners.
Generate new Harris corners after a specific number of frames (say, 10 to 20) to compensate for new points entering the scene or to discard the ones going out of the scene.
Track the new and old Harris points.