Implementing real-time style transfer
In this recipe, we will build our own lightweight style transfer model based on the U-Net architecture. To do so, we will use a dataset generated using Stable Diffusion (see more next about what Stable Diffusion is). This can be seen as a kind of knowledge distillation: we will use the data generated by a large, teacher model (Stable Diffusion, which weighs several gigabytes) to train a small, student model (here, a U-Net++ of less than 30 MBs). This is a funny way to use generative models to create data, but the concepts developed here can be used in many other applications: some will be proposed in the There’s more… section, along with guidance on creating your own style transfer dataset using Stable Diffusion. But before that, let’s give some context about style transfer.
Style transfer is a famous and fun use of deep learning, allowing us to change the style of a given image into another style. Many examples exist, such...
