Now that we know how to recover the motion and scene geometry from two cameras, it would seem trivial to get the parameters of additional cameras and more scene points simply by applying the same process. This matter is in fact not so simple as we can only get a reconstruction that is up-to-scale, and each pair of pictures gives us a different scale.
There are a number of ways to correctly reconstruct the 3D scene data from multiple views. One way is of resection or camera pose estimation, also known as Perspective N-Point(PNP), where we try to solve for the position of a new camera using the scene points we have already found. Another way is to triangulate more points and see how they fit into our existing scene geometry; this will tell us the position of the new camera by means of the Iterative Closest Point(ICP) procedure. In this chapter we will discuss using OpenCV's solvePnP functions to achieve the first method.
The first step we choose in this kind of...