Challenges in training GANs

The challenges mentioned in this section relate to training GANs which can have the following major problems:

• Mode collapse and mode drop
• Training instability
• Sensitivity to hyperparameter initialization
• Vanishing gradients

Let's now move on and address each of these problems individually.

Mode collapse and mode drop

Mode collapse and mode drop are common problems in GANs. They refer to reduced variety in the samples produced by a generator. The term has been used interchangeably and has the following two main interpretations:

1. The generator synthesizes samples with intra-mode variety, but some modes are missing.
2. The generator synthesizes samples with inter-mode variety, but each mode lacks...