On Sept. 26, South Korea revised its law that criminalizes deepfake pornography. Now, it’s not just illegal to create and distribute this lewd digital material, but also to view it. Anyone found to possess, save, or even watch this content could face up to three years in jail or a $22,000 fine.
Deepfakes are AI mashups in which a person’s face or likeness is superimposed onto explicit content without their consent. It’s an issue that’s afflicted celebrities like Taylor Swift, but also private individuals targeted by people they know.
South Korea’s law is a particularly aggressive approach to combating a serious issue. It’s also a problem that’s much older than artificial intelligence itself. Fake nude images have been created with print photo cutouts as far back as the 19th century, but they have flourished in the computer age with PhotoShop and other photo-editing tools. And it’s a problem that’s only been supercharged by the rise and widespread availability of deep learning models in recent years. Deepfakes can be weaponized to embarrass, blackmail, or hurt people — typically, women — whether they’re famous or not.
While South Korea’s complete prohibition may seem attractive to those desperate to eliminate deepfakes, experts warn that such a ban — especially on viewing the material — is difficult to enforce and likely wouldn’t pass legal muster in the United States.
“I think some form of regulation is definitely needed in this space, and South Korea's approach is very comprehensive,” says Valerie Wirtschafter, a fellow at the Brookings Institution. “I do think it will be difficult to fully enforce just due to the global nature of the internet and the widespread availability of VPNs.”
In the US, at least 20 states have already passed laws addressing nonconsensual deepfakes, but they’re inconsistent. “Some are criminal in nature, others only allow for civil penalties. Some apply to all deepfakes, others only focus on deepfakes involving minors,” says Kevin Goldberg, a First Amendment specialist at the Freedom Forum.
“Creators and distributors take advantage of these inconsistencies and often tailor their actions to stay just on the right side of the law,” he adds. Additionally, many online abuses happen across state lines — if not across national borders — making state laws difficult to sue under.
Congress has introduced bills to tackle deepfakes, but none have yet passed. The Defiance Act, championed by Rep. Alexandria Ocasio-Cortez and Sens. Dick Durbin and Lindsey Graham, would create a civil right to action, allowing victims to sue people who create, distribute, or receive nonconsensual deepfakes. It passed the Senate in July but is still pending in the House.
But a full prohibition on sexually explicit deepfakes would likely run afoul of the First Amendment, which makes it very difficult for the government to ban speech — including explicit media.
“A similar law in the United States would be a complete nonstarter under the First Amendment,” says Corynne McSherry, legal director at the Electronic Frontier Foundation. She thinks that current US law should protect Americans from some harms of deepfakes, much of which could be defamatory, an invasion of privacy, or violate citizens’ rights to publicity.
Many states, including California, have a right of publicity law that allows individuals to sue if their likeness is being used without their consent, especially for commercial purposes. For a new law to take action on deepfakes and pass First Amendment scrutiny, it would need to be narrowly tailored to address a very specific harm without infringing on protected speech, something that McSherry says would be very hard to do.
Despite the tricky First Amendment challenges, there is growing recognition of the need for some form of regulation, Wirtschafter says. “It is one of the most pernicious and damaging uses of generative AI, and it disproportionately targets women.”