Improving the 3D tracking algorithm

Essentially, our 3D tracking algorithm combines three approaches:

1. Find a 6DOF pose with a PnP solver, whose inputs depend on FLANN-based matches of ORB descriptors..
2. Use a Kalman filter to stabilize the 6DOF tracking result.
3. If an object was tracked in the previous frame, use a mask to limit the search to the region where the object is now most likely to be found.

Often, commercial solutions for 3D tracking involve additional approaches. We have relied on successfully using a descriptor matcher and a PnP solver for every frame; however, a more complex algorithm may provide some alternatives as fallbacks or as cross-checking mechanisms. This is in case the descriptor matcher and PnP solver miss the object in some frames, or in case they are too computationally expensive to use for every frame. The following alternatives are widely used:

• Update the previous keypoint matches based on optical flow, and update the previous 6DOF pose based on the homography...