What if the images are at an angle to each other?

Until now, we were looking at images that were on the same plane. Stitching those images was straightforward and we didn't have to deal with any artifacts. In real life, you cannot capture multiple images on exactly the same plane. When you are capturing multiple images of the same scene, you are bound to tilt your camera and change the plane. So the question is, will our algorithm work in that scenario? As it turns out, it can handle those cases as well.

Let's consider the following image:

Now, let's consider another image of the same scene. It's at an angle with respect to the first image, and it's partially overlapping as well:

Let's consider the first image as our reference. If we stitch these images using our algorithm, it will look something like this:

If we keep the second image as our reference, it will look something like this:

Why does it look stretched?

If you observe, a portion of the output image corresponding to the query image looks...