Facebook Reality Labs launch SUMO Challenge to improve 3D scene understanding and modeling algorithms

Facebook Reality Labs have launched the Scene Understanding and Modeling SUMO Challenge. The challenge is designed by a group of computer vision researchers at Facebook with collaborators from Stanford, Princeton and Virginia Tech. The goal of the challenge is to aid the development of comprehensive 3D scene understanding and modeling algorithms.

For the SUMO challenge, participants are required to generate an instance-based 3D representation of an indoor scene given only a 360-degree RGB-D image taken from a single viewpoint. The generated scene is modeled by a collection of elements, each of which represents one object, such as a wall, the floor, or a chair.

facebook-reality-labs-launch-sumo-challenge-to-improve-3d-scene-understanding-and-modeling-algorithms-img-0

Source: Facebook Blog

What are the three types of tasks?

The SUMO Challenge is organized into three performance tracks based on the output representation of the scene. Participants can join in any of the three increasingly detailed and difficult performance tracks.
Bounding Boxes Track: The scene is represented by a collection of oriented bounding boxes. This is similar to the SUN RGB-D Object Detection Challenge. The bounding box is the coordinates of the rectangular border that fully encloses a digital image when it is placed over a page, a canvas, or a screen.
Voxels Track: The scene is represented by a collection of oriented voxel grids. A voxel represents a value on a regular grid in three-dimensional space.
Meshes Track: The scene is represented by a collection of textured surface meshes. A mesh is a collection of vertices, edges and faces that defines the shape of a polyhedral object in 3D computer graphics and solid modeling.

How are the tasks evaluated?

The SUMO evaluation metrics focus on four aspects of the representation: geometry, appearance, semantics, and perception (GASP). Participants will be evaluated on their ability to consistently infer the correct geometry, pose, appearance and semantics of the elements in each scene.

The challenge runs from August 29th until November 16th, 2018. The top winners in each track will receive prizes, including cash rewards and NVIDIA Titan X GPUs.