What is epipolar geometry?

Before discussing epipolar geometry, let's discuss what happens when we capture two images of the same scene from two different viewpoints. Consider the following figure:

Let's see how it happens in real life. Consider the following image:

Now, let's capture the same scene from a different viewpoint:

Our goal is to match the keypoints in these two images to extract the scene information. The way we do this is by extracting a matrix that can associate the corresponding points between two stereo images. This is called the fundamental matrix.

As we saw in the camera figure earlier, we can draw lines to see where they meet. These lines are called epipolar lines. The point at which the epipolar lines converge is called epipole. If you match the keypoints using SIFT, and draw the lines towards the meeting point on the left image, it will look like this:

Following are the matching feature points in the right image:

The lines are epipolar lines. If you take the...