Computing the fundamental matrix of an image pair

The previous recipe showed you how to recover the projective equation of a single camera. In this recipe, we will explore the projective relationship that exists between two images that display the same scene. These two images could have been obtained by moving a camera at two different locations to take pictures from two viewpoints or by using two cameras, each of them taking a different picture of the scene. When these two cameras are separated by a rigid baseline, we use the term stereovision.

Getting ready

Let's now consider two cameras observing a given scene point, as shown in the following figure:

We learned that we can find the image x of a 3D point X by tracing a line joining this 3D point with the camera's center. Conversely, the scene point that has its image at the position x on the image plane can be located anywhere on this line in the 3D space. This implies that if we want to find the corresponding point of a given image point...