Generative Fill with Adobe Firefly (Part II)

Adobe Firefly Overview

Adobe Firefly is a new set of generative AI tools which can be accessed via https://firefly.adobe.com/ by anyone with an Adobe ID. To learn more about Firefly… have a look at their FAQ.  

Image 1: Adobe Firefly

For more information about the usage of Firefly to generate images, text effects, and more… have a look at the previous articles in this series:

•       Animating Adobe Firefly Content with Adobe Animate
•       Exploring Text to Image with Adobe Firefly
•       Generating Text Effects with Adobe Firefly
•        Adobe Firefly Feature Deep Dive
•        Generative Fill with Adobe Firefly (Part I)

This is the conclusion of a two-part article. You can catch up by reading Generative Fill with Adobe Firefly (Part I). In this article, we’ll continue our exploration of Firefly with the Generative fill module by looking at how to use the Insert and Replace features… and more.

Generative Fill – Part I Recap

In part I of our Firefly Generative fill exploration, we uploaded a photograph of a cat, Poe, to the AI and began working with the various tools to remove the background and replace it with prompt-based generative AI content.

Image 2: The original photograph of Poe

Note that the original photograph includes a set of electric outlets exposed within the wall. When we remove the background, Firefly recognizes that these objects are distinct from the general background and so retains them.

Image 3: A set of backgrounds is generated for us to choose from

You can select any of the four variations that were generated from the set of preview thumbnails beneath the photograph.

Again, if you’d like to view these processes in detail – check out Generative Fill with Adobe Firefly (Part I).

Insert and Replace with Generative Fill

We covered generating a background for our image in part I of this article. Now we will focus on other aspects of Firefly Generative fill, including the Remove and Insert tools.

Consider the image above and note that the original photograph included a set of electric outlets exposed within the wall. When we removed the background in part I, Firefly recognized that they were distinct from the general background and so retained them. The AI has taken them into account when generating the new background… but we should remove them.

This is where the Remove tool comes into play.

Image 4: The Remove tool

Switching to the Remove tool will allow you to brush over an area of the photograph you’d like to remove. It fills in the removed area with pixels generated by the AI to create seamless removal.

1.               Select the Remove tool now. Note that when switching between the Insert and Remove tools, you will often encounter a save prompt as seen below. If there are no changes to save, this prompt will not appear!

Image 5: When you switch tools… you may be asked to save your work

2.               Simply click the Save button to continue – as choosing the Cancel button will halt the tool selection.

3.               With the Remove tool selected, you can adjust the Brush Settings from the toolbar below the image, at the bottom of the screen.

Image 6: The Brush Settings overlay

4.               Zoom in closer to the wall outlet and brush over the area by clicking and dragging with your mouse. The size of your brush, depending upon brush settings, will appear as a circular outline. You can change the size of the brush by tapping the [ or] keys on your keyboard.

Image 7: Brushing over the wall outlet with the Remove tool

5.               Once you are happy with the selection you’ve made, click the Remove button within the toolbar at the bottom of the screen.

Image 8: The Remove button appears within the toolbar

6.               The Firefly AI uses Generative fill to replace the brushed-over area with new content based upon the surrounding pixels. A set of four variations appears below the photograph. Click on each one to preview – as they can vary quite a bit.

Image 9: Selecting a fill variant

7.               Klick the Keep button in the toolbar to save your selection and continue editing. Remember – if you attempt to switch tools before saving… Firefly will prompt you to save your edits via a small overlay prompt.

The outlet has now been removed and the wall is all patched up.

Aside from the removal of objects through Generative fill, we can also perform insertions based on text prompts. Let’s add some additional elements to our photograph using these methods.  

1.               Select the Insert tool from the left-hand toolbar.

2.               Use it in a similar way as we did the Remove tool to brush in a selection of the image. In this case, we’ll add a crown to Poe’s head – so brush in an area that contains the top of his head and some space above it. Try and visualize a crown shape as you do this.

3.               In the prompt input that appears beneath the photograph, type in a descriptive text prompt similar to the following: “regal crown with many jewels”

Image 10: A selection is made, and a text prompt inserted

4.               Click the Generate button to have the Firefly AI perform a Generative fill insertion based upon our text prompt as part of the selected area.

Image 11: Poe is a regal cat

5.               A crown is generated in accordance with our text prompt and the surrounding area. A set of four variations to choose from appears as well. Note how integrated they appear against the original photographic content.

6.               Click the Keep button to commit and save your crown selection.

7.               Let’s add a scepter as well. Brush the general form of a scepter across Poe’s body extending from his paws to his shoulder.

8.               Type in the text prompt: “royal scepter”

Image 12: Brushing in a scepter shape

9.               Click the Generate button to have the Firefly AI perform a Generative fill insertion based upon our text prompt as part of the selected area.

Image 13: Poe now holds a regal scepter in addition to his crown

10.            Remember to choose a scepter variant and click the Keep button to commit and save your scepter selection.

Okay! That should be enough regalia to satisfy Poe. Let’s download our creation for distribution or use in other software.

Downloading your Image

Click the Download button in the upper right of the screen to begin the download process for your image.

Image 14: The Download button

As Firefly begins preparing the image for download, a small overlay dialog appears.

Image 15: Content credentials are applied to the image as it is downloaded

Firefly applies metadata to any generated image in the form of content credentials and the image download process begins.

Once the image is downloaded, it can be viewed and shared just like any other image file.

Image 16: The final image from our exploration of Generative fill

Along with content credentials, a small badge is placed upon the lower right of the image which visually identifies the image as having been produced with Adobe Firefly.

That concludes our set of articles on using Generative fill to remove and insert objects into your images using the Adobe Firefly AI. We have a number of additional articles on Firefly procedures on the way… including Generative recolor for vector artwork!

Author Bio

Joseph Labrecque is a Teaching Assistant Professor, Instructor of Technology, University of Colorado Boulder / Adobe Education Leader / Partner by Design

Joseph is a creative developer, designer, and educator with nearly two decades of experience creating expressive web, desktop, and mobile solutions. He joined the University of Colorado Boulder College of Media, Communication, and Information as faculty with the Department of Advertising, Public Relations, and Media Design in Autumn 2019. His teaching focuses on creative software, digital workflows, user interaction, and design principles and concepts. Before joining the faculty at CU Boulder, he was associated with the University of Denver as adjunct faculty and as a senior interactive software engineer, user interface developer, and digital media designer.

Labrecque has authored a number of books and video course publications on design and development technologies, tools, and concepts through publishers which include LinkedIn Learning (Lynda.com), Peachpit Press, and Adobe. He has spoken at large design and technology conferences such as Adobe MAX and for a variety of smaller creative communities. He is also the founder of Fractured Vision Media, LLC, a digital media production studio and distribution vehicle for a variety of creative works.

Joseph is an Adobe Education Leader and member of Adobe Partners by Design. He holds a bachelor’s degree in communication from Worcester State University and a master’s degree in digital media studies from the University of Denver.

Author of the book: Mastering Adobe Animate 2023