Exploring the uses of deepfakes

The original use of Deepfakes might be the one that required the least amount of imagination. Putting one person’s face on another’s person has many different uses in various fields. Please don’t consider the ideas here as the full extent of the capabilities of deepfakes – someone is bound to imagine something new!

Entertainment

Entertainment is the first area that comes to mind for most people when they consider the usage of deepfakes. There are two main areas of entertainment in which I see deepfakes playing a significant role: narrative and parody.

Narrative

The utility of deepfakes in movies is obvious. Imagine an actor’s face being superimposed onto their stunt double or an actor who becomes unavailable being replaced by another performer without any changes to the faces in the final movie.

While deepfakes may not seem good enough, deepfakes are already being used in Hollywood and other media today – from Detective Pikachu, which used deepfakes to de-age Bill Nighy, to For All Mankind, which used it to put actors face to face with Ronald Reagan. Agencies and VFX shops are all examining how to use deepfakes in their work.

These techniques are not unique to deepfakes. CGI (in this book, referring to 3D graphics) face replacements have been used in many movies. However, using CGI face replacement is expensive and complicated, requiring filming to be done in particular ways with lots of extra data captured to be used by the artists to get the CGI face to look good in the final scene. This is an art more than a science and requires extensive skills and knowledge to accomplish. Deepfakes solve many of these problems making new forms of face replacements possible.

Making a deepfake requires no special filming techniques (although some awareness will make the process smoother). Deepfakes also require very little attention or skill compared to CGI face replacements. This makes it ideal for lower-cost face replacements, but it can also be higher-quality since the AI accounts for details that even the most dedicated artist can’t recreate.

Parody

Parody is an extremely popular form of social criticism and forms the basis for entire To PD: movies, TV shows, and other forms of media. Parody is normally done by professional impersonators. In some cases, those impersonators look (or can be made to look) similar to the person they’re impersonating. Other times, there is a reliance on their performance to make the impersonation clear.

Deepfakes provide an opportunity to change the art of parody wherein the impersonator can be made to look like the individual being parodied via a deepfake instead of by chance of birth. By removing the attention from basic appearance, deepfakes allow the focus to be placed directly on the performance itself.

Deepfakes also enable a whole new form of parody in which normal situations can become parodic simply due to the changed face. This particular form becomes humorous due to the distinct oddity of very different faces, instead of an expected swap.

Figure 1.1 – Steve Buscemi as Jennifer Lawrence by birbfakes

Note

This image is included with the kind permission of its original creator, birbfakes. You can view the original video here: https://youtu.be/r1jng79a5xc.

Video games

Video games present an interesting opportunity when it comes to deepfakes. The idea here is that a computer-generated character could be deepfaked into a photorealistic avatar. This could be done for any character in the game, even the player’s character. For example, it would be possible to make a game in which, when the player’s character looked into a mirror, they would see their own face looking back at them. Another possibility would be to replace a non-player character with a deepfake of the original actor, allowing for a far more realistic appearance without making a complete 3D clone of the actor.

Education

Education could also benefit from deepfakes. Imagine if your history class had a video of Abraham Lincoln himself reading the Gettysburg address. Or a corporate training video in which the entire video is hosted by the public mascot (who may not even be a real person) without having to resort to costumes or CGI. It could even be used to allow multiple videos or scenes filmed at significantly different times to appear to be more cohesive by appearing to show the actor at the same time.

Many people are very visual learners and seeing a person “come alive” can really bring the experience home. Bringing the pre-video past to life using deepfakes enables a whole new learning experience. One example of this is the Dalí Museum, which created a series of videos of Salvador Dalí talking to guests. This was done by training a deepfake model on an actor to put Dalí’s face on the videos. Once the model was trained and set up, they were able to convert many videos, saving a lot of time and effort compared to a CGI solution.

Advertisements

Advertising agencies are always looking for the newest way to grab attention and deepfakes could be a whole new way to catch viewers’ attention. Imagine if you walked past a clothing store, you stopped to look at an item of clothing in the window, and suddenly the screen beside the item showed a video of an actor wearing the item but with your face, allowing you to see how the item would look on you. Alternatively, a mascot figure could be brought to life in a commercial. Deepfakes offer a whole new tool for creative use, which can grab attention and provide whole new experiences in advertising.

Now that we’ve got some idea of a few potential uses for deepfakes, let’s take a quick look under the hood and see how they work.