Understanding ray sampling

Ray sampling is the process of emitting rays from the camera that goes through the image pixels and sampling points along these rays. The ray sampling scheme depends on the use case. For example, sometimes we might want to randomly sample rays that go through some random subset of image pixels. Typically, we need to use such a sampler during training since we only need a representative sample from the full data. In such cases, we can use MonteCarloRaysampler in Pytorch3D. In other cases, we want to get the pixel values for each pixel on the image and maintain a spatial order. This is useful for rendering and visualization. For such use cases, PyTorch3D provides NDCMultiNomialRaysampler.

In the following, we will demonstrate how to use one of the PyTorch3D ray samplers, NDCGridRaysampler. This is like NDCMultiNomialRaysampler, where pixels are sampled along a grid. The codes can be found in the GitHub repository named understand_ray_sampling.py:

...