Refinement of the reconstruction

One of the most important parts of an SfM method is refining and optimizing the reconstructed scene, also known as the process of Bundle Adjustment (BA). This is an optimizing step where all the data we gathered is fitted to a monolithic model. Both the position of the 3D points and the positions of cameras are optimized, so reprojection errors are minimized (that is, approximated 3D points are projected on the image close to the position of originating 2D points). This process usually entails the solving of very big linear equations of the order of tens of thousands of parameters. The process may be slightly laborious, but the steps we took earlier will allow for an easy integration with the bundle adjuster. Some things that seemed strange earlier may become clear; for example, the reason we retain the origin 2D points for each 3D point in the cloud.

One implementation of a bundle adjustment algorithm is the Simple Sparse Bundle Adjustment (SSBA) library...