Deepfakes in the 2024 US Presidential Election


maintained by Hany Farid

Deepfakes (aka synthethic media or generative AI) have captured the imagination of some and struck fear in others. Deepfakes -- image, audio, or video automatically synthesized by a machine-learning system -- are the latest in a long line of techniques used to manipulate reality. Their introduction, however, poses new risks due to the democratized access to sophisticated manipulation tools.

Of particular concern is how deepfakes will be weaponized in politics. Here we keep track of known examples of deepfakes in the upcoming 2024 US presidential election.

Please email me (hfarid@berkeley.edu) if you have supporting evidence that any of the claims below are inaccurate, or if you have additional examples.

Fred Trump


February 18, 2024 | manipulated video

content: On the heals of Donald Trump accusing some in the media of using AI to create unflattering footage of him, the Lincoln Project (a centrist PAC), released an AI-generated video of Trump's father, Fred Trump saying things like "Everyone knows why the women leave you, you're boring, broke, and I told you not to get fat. You pay porn stars and you are weak down there. Your kids hate you, especially the girl. How did a son of mine turn out so damn dumb? The brand I built is crap because of you. You're trash. You'll be lucky to stay out of jail."

analysis: The video contains the disclosure "This video has been manipulated by technical means and depicts speech or conduct that did not occur."

fact check: Futurism
An AI-gnerated Fred Trump

Trump's Bloody Hand


February 1, 2024 | photo in question

content: As Trump was leaving an event on Jan. 17th of this year, he waved to reporters. The video shows what appears to be bruises/blood on his right hand. When he was asked later by Fox News reporter Mark Meredith “How’s your hand? It looks better now,” Trump responded "My hand?" After some back and forth Trump responded "Nothing. Maybe it was AI."

analysis: Because this photo was taken from a longer video, throughout which the bruises are visible, it is unlikely that this is an AI-generated photo or a more traditionally manipulated photo. In addition, our model trained to distinguish real from AI-generated images classifies this image as unlikely to be AI-generated. This may be an example of the liar's dividend where in the age of deepfakes it is has become increasingly easier to claim that something is fake.

fact check: The Hill
XXX

Republican Victory in 2024


February 1, 2024 | manipulated images [ 1 | 2 ]

content: Posted on twitter.com by Patrick Ruffini and racking up over 2 million views, these images were accompanied with the tweet "Republican victory in 2024 depends on their being able to do this."

analysis: These images are AI-generated. The man on the right in the first image has three arms and the text on the hat and shirt are gibberish -- common artifacts of today's generative AI. Similarly, the text in the second image is gibberish, and the American flag in the foreground appears to have only one star. In addition, our model trained to distinguish real from AI-generated images classifies these images as highly likely to be AI-generated.

fact check: this one is easy
An AI-generated image showing a person of color wearing red with a clipboard talking to another person of color.

Biden in Fatigues


January 30, 2024 | manipulated images [ 1 | 2 | 3 ]

content: A series of photos circulated on twitter.com purportedly showing President Biden in military fatigues on the verge of authorizing military strikes.

analysis: The image is AI-generated. There are tell-tale signs of misinformed objects on the table, and our geometric analysis of the ceiling tiles reveals a physically inconsistent vanishing point.

fact check: AFP Fact Check
An AI-generated image of President Biden purportedly wearing military fatigues.

What a Bunch of Malarkey


January 22, 2024 | manipulated audio

content: In the lead up to the New Hampshire Democratic primary, voters in the state received robocalls impersonating President Biden. The call urged voters not to vote in the primary and to "save your vote for the November election." The call went on to say "voting this Tuesday only enables the Republicans in their quest to elect Donald Trump again. Your vote makes a difference in November, not this Tuesday." It is currently unclear who is behind these calls.

analysis: The quality of the deepfake voice is quite low and it is fairly clear from the voice and cadence that it is AI-generated. In addition, our models, trained to distinguish real from fake audio, classifies, with high confidence, this audio has AI-generated.

fact check: NBC News

update: New Hampshire Attorney General John Formella announced on Feb 6, 2024 that his office has identified two businesses beind these robocalls, Life Corporation and Lingo Telecom. Formella said "We have issued a cease-and-desist letter to Life Corporation that orders the company to immediately desist violating New Hampshire election laws. We have also opened a criminal investigation, and we are taking next steps in that investigation, sending document preservation notices and subpoenas to Life Corporation, Lingo Telecom, and any other individual or entity" [source]
An AI-generated voice of President Joe Biden

An AI-generated Rep. Dean Phillips


January 18, 2024 | manipulated audio

content: A super PAC supporting Rep. Dean Phillips' campaign created an interactive AI-powered version of Rep. Phillips. The bot was powered by OpenAI's ChatGPT and AI-powered voice cloning/synthesis. OpenAI forbids the use of its tools in political campaigns and shortly after the bot's release, OpenAI suspsended the creator's account. This is not the first AI-powered candidate; A similar interactive AI-bot was created of Mayor Suarez (see July 8, 2023 entry below).

analysis: The bot creators and the bot itself acknowledge it is AI-generated.

fact check: Washington Post and Washington Post
An AI-generated voice of Rep. Dean Phillips

Biden Sending F15s to Texas


January 13, 2024 | manipulated audio

content: As tensions flared between Texas and the US Government over access to the U.S.-Mexico border, this audio spread across TikTok claiming to be President Biden saying "We’re going to make sure those cowboys don’t stop the surge of military-age men from entering. If we have to send F-15s to Texas here and wage war against Texas, so be it." Similar videos spread on twitter.com.

analysis: The White House claims the audio is fake, and there is no public record of President Biden saying anything to this effect. The audio has tell-tale signs of being AI-generated in terms of the cadence of the speech.

fact check: PolitiFact
A screenshot of a fake/video/audio of President Biden saying he is going to send F15s to Texas

Trump and Epstein


January 4, 2024 | manipulated image

content: Shortly after a Judge unsealed documents in the case of convicted child sex trafficker Jeffrey Epstein (who died in jail in 2019), this photo began circulated on twitter.com before circulated more broadly on various other platforms. The photo purports to show Donald Trump, Epstein and a young girl.

analysis: There are many tell-tale signs that this is an AI-generated photo, including the distorted fingers of the young girl as well as the distorted faces of the people in the background. In addition, our model trained to distinguish real from AI-generated images classifies this image as highly likely to be AI-generated.

fact check: Snopes.com
An AI-generated image of Donald Trump, Jeffrey Epstein and a young girl.

What Ron DeSantis Should Say But Won't


September 1, 2023 | manipulated video

content: YouTube creator C3PMeme made this video of Ron DeSantis saying, in part, "I've realized that I need to drop out of this race immediately. I was told I could be President by some very powerful people and I took some very bad advice... I never should have challenged President Trump, the greatest president of my life time... It's time to right this ship and help President Trump get reelected to make America great again."

analysis: From the fairly crude mouth motion and audio quality to the video description "A deepfake of what Ron DeSantis might say if he dropped out of the race (which he should)", this video is obviously a fake and was not created to be deceptive. Based on our forensic analysis, we find that the voice in this video is highly likely to be AI-generated.

fact check: Forbes
An image of Ron DeSantis at his desk

Trump Attacks Iowa


July 17, 2023 | manipulated video

content: A PAC supporting Ron DeSantis created an ad attacking former President Donald Trump. The ad takes issue with Trump's attack on Iowa Governor Kim Reynolds. In the video ad, we hear Trump's voice criticizing Reynolds. The text appears to be based on a post by Trump on Truth Social. The ad is scheduled to run in Iowa with at least a $1 million ad buy.

analysis: A person familiar with the ad confirmed that Trump's voice was AI generated.

fact check: Politico | PolitiFact
A political ad containing an AI-synthesized voice of Donald Trump

Ask AI Suarez


July 8, 2023 | Ask AI Suarez

content: A super PAC supporting Miami Mayor Francis Suarez's presidential run launched an interactive AI chatbot to engage with voters. Suarez's campaign denies being involved in creating this AI bot or what their PAC does.

analysis: The content is clearly labeled as AI generated. It appears that the bot mostly listens to a user's question and then directs them to a relevant stock video response.

fact check: AP News
An AI version of Francis Suarez

Trump or Biden 2024


July 8, 2023 | manipulated video

content: AI-generated versions of Donald Trump and Joe Biden engaged in a user-prompted debate is streaming 24/7 on Twitch. The content is often riddled with offensive language and appears to be powered by a chatGPT-like language model for crafting the script, voice cloning, and a lip-sync video manipulation.

analysis: The content is clearly labeled as AI generated and parody.
An image of Trump and Biden sharing the stage in their AI-generated fake debate

Trump Dancing with Young Girl


June 14, 2023 | manipulated image

content: An image purportedly showing Donald Trump dancing with a young girl spread on Twitter and TikTok. This image was not associated with a political advertisement.

analysis: The image contains physical inconsistencies consistent with AI generation including a misformed hand of the man in the background with six fingers grasping a glass of beer. The girl's left eye contains a specular reflection consistent with, for example, a camera flash, but her right eye improbably does not contain the same reflection.

Trump's campaign accused the DeSantis campaign which has previously created fake images of Trump (see below). The DeSantis campaign denied the allegation.

fact check: Newsweek | AP News
A fake photo of Donald Trump dancing with a young girl

Leaked Trump Tape


June 11, 2023 | manipulated audio

content: This audio was created by Matthew Ellis with full disclosure that it is a deepfake. The description on YouTube reads "This is a DEEP FAKE. The purpose of this video it is to illustrate the severity of Trump's actions listed in the Indictment and to show how AI and Fake Audio/Video recordings need to be flagged by Platforms and Content Creators."

analysis: This audio is clearly labeled as a deepfake. It is included here to show the quality of AI-cloned voices.

fact check:
A screenshot of a YouTube video that transcribes the fake lekaed Trump tape

DeSantis Campaign Ad


June 7, 2023 | manipulated images [00:24-00:30 sec]

content: A DeSantis campaign video was releaased attacking Donald Trump by showing, in part, a montage of photos of Trump embracing Dr. Fauci.

analysis: Three of the photos in the six-photo montage (top-left, bottom-center, and bottom-right) show clear signs of AI-generatation. Each of these images contain physical inconsistencies including: the text on the White House logo is nonsensical (top-left), the stars on the flag are misshapen (bottom-center), and the shape of Trump's ear is inconsistent with reference images (bottom-right).

fact check: Agence France-Presse (AFP)
a fake image of President Trump embracing Dr. Fauci

DeSantis and Musk on Twitter


May 23, 2023 | manipulated audio

content: Former President Donald Trump released an AI-generated clip of a fake Twitter Spaces event consisting of a conversation between Elon Musk, George Soros, Klaus Schwab, an incessantly coughing Dick Cheney, Satan, and Adolf Hitler. This clip was released after the glitchy Twitter Spaces launch of DeSantis' presidential announcement.

analysis: The addition of Hitler and The Devil makes it obvious that this audio is fake.

fact check: Gizmodo
a screenshot

Harris Rambling Speech


May 4, 2023 | manipulated video

content: A recorded speech originally given by Vice President Harris on April 25, 2023 at Howard University was digitally altered to replace the original voice track with a seemingly inebriated and rambling voice saying "Today is today, and yesterday was today yesterday, tomorrow will be today tomorrow so live today so the future today will be as the past today as it is tomorrow."

analysis: There are clear signs of audio/video misalignment. The original video was eventually released showing the same clothing and background, and the impersonated audio was traced back to a TikTok user @mommyrn88. It is not entirely clear if this is a lipsync deepfake or just a classic audio/video dubbing.

fact check: Reuters
Vice\
 President Harris at the microphone

RNC's Biden Ad


April 25, 2023 | manipulated video

content: Shortly after President Biden announced his bid for reelection, the Republican National Committee (RNC) released a video ad featuring a series of fake images. The video imagines "what if crimes worsens," "what if international tensions escalate," and "what if the financial system crumbles," each with an associated fake image showing the corresponding apocalyptic future.

analysis: A small disclaimer in the top left of the frame says "built entirely with AI imagery," and the caption underneath the YouTube video reads, "An AI-generated look into the country's possible future if Joe Biden is re-elected in 2024."

fact check: The Verge
A single frame of an R\
NC ad showing a dystopian San Francisco with the title 'What if crime worsens'

Trump Praying


March 24, 2023 | manipulated image

content: Weeks before the first indictment of former President Donald Trump, a series of fake images circulated online showing his arrest. In response, Trump shared his own fake image on Truth Social showing him kneeling in prayer.

analysis: Initial versions of AI-generated images struggled to create realistic and physically consistent hands. In this image, Trump's fingers are misformed. The faces of the men in the background are also highly distorted, consistent with physical inconsistencies made by generative AI systems.

fact check: BBC
A fake photo of Donald Trump praying

Biden on Banking Turmoil


March 15, 2023 | manipulated audio

content: Following the collapse of Silicon Valley Bank and the resulting financial instability, an audio clip purportedly of President Biden was released in which he is heard saying, in part, "all the money is gone" and "a collapse is imminent."

analysis: When separated from the recording, the background noise has a highly regular pattern that appears to be synthetic and not the result of a physically noisy system. The cadence of the speech is highly regular, consistent with synthetically-generated speech and atypical of natural speech.

fact check: Associated Press
President Biden at the microphone

Sen. Warren: No Voting for Republicans


February 27, 2023 | manipulated video

content: A video of Senator Elizabeth Warren on MSNBC appears to show her saying that Republicans should be barred from voting in the 2024 presidential election. This videos appears to have originated on TikTok and eventually made its way to Twitter where it received much more engagement.

analysis: The video is a modified version of a December 2022 appearance Warren made on MSNBC.

fact check: Newsweek
Senator Warren being interviewed on MSNBC