Challenges and limitations of photorealistic synthetic data

In this section, you will explore the main challenges that hinder generating photorealistic synthetic data in practice. We will highlight the following limitations.

Creating hyper-realistic scenes

The real world is complex, diverse, and intricate with details. Scene elements in reality have various shapes, sophisticated dynamics, and highly non-linear interactions. Additionally, our vision and perception of the world are limited and subject to many factors, such as cognitive biases and color perception. Additionally, we may judge photorealism differently based on the context and evaluator. For example, what is more photorealistic, realistic foreground objects and a non-realistic background or the opposite? All these aspects together make generating highly realistic scenes rather hard in practice.

Resources versus photorealism trade-off

Budget, time, skills, and other factors limit the photorealism of the generated...