Overlaying the depth frame over the image frame

The depth and color streams are from two different sensors; because of the difference in their positions in the device, they see objects from two different angles. This makes it hard to figure out which two pixels in these two streams are related to each other in the physical world. Fortunately, this problem is solved by OpenNI using one of its built-in methods. Using this feature, a programmer can expect each pixel of depth to be in the same position as its color pair. This feature is very useful for different types of projects, including, but not limited to, generating a color point cloud.

Here, we try to show you how to overlay the depth data over the color data using this feature of OpenNI.

Getting ready

Create a project in Visual Studio 2010 and prepare it for working with OpenNI using the Creating a project in Visual Studio 2010 recipe of Chapter 2, OpenNI and C++. Then configure Visual Studio 2010 to use OpenGL with the Configuring Visual...