Generating synthetic data

In this section, we will learn how to generate synthetic data using modern game engines such as Unity and Unreal.

To generate synthetic data with its corresponding ground truth, it is recommended that we follow these steps:

1. Identify the task and ground truth to generate.
2. Create the 3D virtual world in the game engine.
3. Set the virtual camera.
4. Add noise and anomalies.
5. Set the labeling pipeline.
6. Generate the training data with the ground truth.

Throughout this section, we will thoroughly discuss each facet of this process.

Identify the task and ground truth to generate

The first step in the synthetic data generation process is defining the task, the type of the data, and the ground truth to generate. For example, the data could be images, videos, or audio. At the same time, you need to identify what ground truth to generate for your problem. For example, you can generate semantic segmentation, instance segmentation...