Understanding 3D image tracking and augmented reality

We have already solved problems involving image matching in Chapter 6, Retrieving Images and Searching Using Image Descriptors. Moreover, we have solved problems involving continuous tracking in Chapter 8, Tracking Objects. Therefore, we are familiar with many of the components of an image tracking system, though we have not yet tackled any 3D tracking problems.

So, what exactly is 3D tracking? Well, it is the process of continually updating an estimate of an object's pose (its position and orientation) in a 3D space. Typically, the pose is expressed in terms of six variables: three variables to represent the object's 3D translation (that is, position) and the other three variables to represent its 3D rotation (that is, orientation).

A more technical term for 3D tracking is 6DOF tracking – that is, tracking with 6 degrees of freedom, meaning the 6 variables we just mentioned. With any fewer than 6 variables, it would...