Object and style transferring
When we introduced the theory behind SD in Chapters 4 and 5, we learned that only text embedding is involved in the UNet diffusion process. Even if we provide an initial image as the starting point, the initial image is simply used as the starting noise or concatenated with initial noises. It does not have any influence on the steps of the diffusion process.
That is until the IP-Adapter project [3] came about. IP-Adapter is a tool that lets you use an existing image as a reference for text prompts. In other words, we can take the image as another piece of prompt work together with text guidance to generate an image. Unlike Textual Inversion, which usually works well for certain concepts or styles, IP-Adapter works with any images.
With the help of IP-Adapter, we can magically transfer an object from one image to a completely different one.
Next, let’s start using IP-Adapter to transfer an object from one image to another one.
