China is cracking down on deepfakes at home while profiting from them abroad.
Deepfake apps from China-based developers have been downloaded millions of times in the U.S. and remain available on app stores even after China instituted strict laws this year regulating deepfake videos on its own internet.
The apps allow Chinese developers to profit from data and dollars brought in through America’s largely unregulated tech market, while U.S. lawmakers have focused on other targets, such as TikTok. That scrutiny has not been extended to other China-based apps, such as the online marketplaces Shein and Temu, which are popular in the U.S. and other parts of the world.
Deepfakes are manipulated media, often created with artificial intelligence technology, in which faces, appearances and voices have been altered. The technology has improved in recent years and has been used to make viral videos, some to warn about the technology’s ability to create misinformation.
But more than anything else, deepfake technology has fueled an online economy centered on creating fake pornography. Sensity, an Amsterdam-based company that detects and monitors AI-developed synthetic media for industries like banking and fintech, found that 96% of deepfakes are sexually explicit and feature women who did not consent to their creation.
An NBC News review of free deepfake apps on Apple’s App Store and the Google Play store found more than 17 deepfake apps available to download from companies based all over the world, including Russia, Ukraine, the U.K., the U.S., Italy and China.
Some of the apps reviewed by NBC News, such as the popular Cyprus-based FaceApp and China-based Facee, do not offer features that allow the production of pornographic videos. Such apps are meant to alter images of faces with preset filters. But other apps specifically market their abilities to create nonconsensual deepfake pornography.
One of the China-based apps, FaceMagic, is running sexually explicit ads that say “Make deepfake porn in a sec” on the most popular deepfake porn website, according to a review of the advertisements by NBC News. It has been downloaded more than 2.4 million times since May 2021, according to data from Apptopia, a company that tracks the app market.
Another China-based deepfake app, FaceMega, was recently removed from the Apple and Google app stores after NBC News found that it ran more than 260 ads on Facebook depicting Emma Watson’s likeness in a sexually suggestive nature.
Those apps would almost certainly run afoul of China’s new deepfake rules, which went into effect in January. The rules prohibit creating deepfakes without the consent of the people whose likenesses are being manipulated, and they require labeling deepfakes as AI-generated.
The U.S. has no such federal laws, although some states impose civil penalties for creating nonconsensual porn and deepfakes that depict political disinformation. Some privacy activists have pushed for new rules around deepfakes, but most politicians have instead focused on TikTok.
“In the same way that we are well behind on the regulation of privacy, I think we’re still very much at the early stages of determining what regulation we should have around deepfakes,” said Samir Jain, the director of policy at the Center for Democracy and Technology, a nonprofit that advocates for digital rights and free expression.
Concerns about foreign access to U.S. user data through apps have coalesced in recent years around China and TikTok. Critics of TikTok have argued that it poses a particularly urgent security threat, because China could force it to divulge data about its U.S. users.
TikTok CEO Shou Zi Chew told a congressional hearing last month that ByteDance, TikTok’s Chinese parent, was a private company and not controlled by the Chinese government.
“TikTok has never shared, or received a request to share, U.S. user data with the Chinese government,” he said in prepared remarks. “Nor would TikTok honor such a request if one were ever made.”
China, which says it would “firmly oppose” any forced sale of TikTok in the U.S., says it takes data privacy and security very seriously and that the U.S. has provided no evidence that TikTok threatens its national security.
“The Chinese government has never asked and will never ask any company or individual to collect or provide data, information or intelligence located abroad against local laws,” Foreign Ministry spokesperson Mao Ning said last month.
An request for comment sent to the Chinese embassing in the U.S. was not answered.
Other countries have taken broad steps to address data privacy issues, as well as concerns about the spread of deepfakes. The European Union has strict privacy rules that it has enforced against many major tech companies, including Amazon, Meta and Google. The E.U. also created rules meant to force major tech platforms to limit the spread of deepfakes.
None have gone as far as China with its recent deepfake rules.
“China as a hub for AI development is increasingly seeking to export its own vision of what AI governance means,” said Michael Karanicolas, the executive director of the UCLA Institute for Technology Law & Policy. “For all the focus on Chinese apps’ disrupting the American economy, China is saying we need rules around this stuff.”
Mary Anne Franks, a law professor and the president of the Cyber Civil Rights Initiative, a nonprofit group that combats nonconsensual porn, said that the commercial model for nearly every app operating in the U.S. involves selling user data and that those legal vulnerabilities overlap with the legal gray area that allows the deepfake economy to flourish.
Deepfake porn websites are a recently growing phenomenon, with search interest trending historically high early this year.
“If the actual powers that be had paid attention to this crisis and anticipated that one day the technology wasn’t going to be crude and wasn’t going to be distinguishable, we wouldn’t be in this position,” Franks said.
Franks said there are legitimate complaints about TikTok, as there are for many other apps. Deepfake apps also collect user data, which can be shared and sold with anyone.
That information, however, is not particularly hard to find. And banning TikTok would not change much.
“If China is interested in that information, they can buy it freely on the secondary market,” Karanicolas said. “You can ban an app, but it doesn’t really affect the broader problem.”