What is ControlNet and how is it different?
In terms of “control,” you may recall textual embedding, LoRA, and the image-to-image diffusion pipeline. But what makes ControlNet different and useful?
Unlike other solutions, ControlNet is a model that works on the UNet diffusion process directly. We compare these solutions in Table 13.1:
|
Control Method |
Functioning Stage |
Usage Scenario |
|
Textual Embedding |
Text encoder |
Add a new style, a new concept, or a new face |
|
LoRA |
Merge LoRA weights to the UNet model (and the CLIP text encoder, optional) |
Add a set of styles, concepts, and generate content |
|
Image-to-Image |
Provide the initial latent image |
Fix... |
