In-painting
In-painting is the task of replacing a certain portion of an image with another image. An example of in-painting is as follows:
Figure 17.1: The first three items—image, mask_image, and prompt—serve as the inputs, while the rightmost image represents the output of the inpainting process.
In the preceding image, we provide the mask corresponding to the subject that we want to replace – a dog. Additionally, we provide the prompt that we want to use to generate an image. Using the mask and prompt, we should generate an output that satisfies the prompt while keeping the rest of the image intact.
An in the following section, we will understand the model training workflow of in-painting.
Model training workflow
In-painting model is trained as follows:
- The input requires an image and a caption associated with the input.
- Pick a subject (a dog in Figure 17.1) that is mentioned in the caption and obtain a mask corresponding...
