Additional topics
In this chapter, we worked only with objects in screen coordinates, by just using depth values of depth images. But in many applications, you need to use the 3D point cloud, which represents the depth image points in 3D space—in the world coordinate system centered at the camera. For obtaining a 3D point cloud from the depth image, you should convert all its pixels to 3D points using the openNIDevice.projectiveToWorld( p ) function. Namely, each pixel (x, y) should be represented as point p, where p.x = x, p.y = y, and p.z is equal to the value of the depth image's pixel (x, y): p.z = openNIDevice.getDepthRawPixels()[ x + w * y ].
The 3D point cloud can be used for:
Accurately modeling the distance distribution over the flat surface. See the discussion at the end of the Creating interactive surface section.
Making dynamic video mapping, which maps a projector image exactly on a moving object such as a performer. To achieve this, you should use geometrical information of relative...
