China regulates deepfake technology

China will begin enforcing its strict new rules around the creation of deepfakes from today, 11 January 2023. 

Deepfakes are images and videos which combine mixed source material to produce a synthetic result. They allow users to replace one person's face with another in a video, or to put words into a speaker's mouth, with their ability to pass as convincing realities sometimes outpacing the progress of tools to spot them effectively.

In March 2022, a poor-quality deepfake of President Volodymyr Zelensky announcing Ukraine’s surrender to Russia surfaced on social media for a brief round of ridicule, before being removed.

Due to their realism, deepfakes have come into question and sparked disinformation fears around the globe.

In December 2022, the Cyberspace Administration of China (CAC) announced it had finalised a set of rules governing “deep synthesis technologies”, which it identified as a "danger to national security and social stability".

The new regulations require businesses offering deepfake services to obtain the real identities of their users and ensure that their AI algorithms are not being misused for illegal activities, like committing fraud, scams, or distributing false and harmful information.

They also require deepfake content to be appropriately labeled to avoid "any confusion" on the part of the public. 

“In recent years, in-depth synthetic technology has developed rapidly. While serving user needs and improving user experiences, it has also been used by some criminals to produce, copy, publish, and disseminate illegal and bad information, defame, detract from the reputation and honour of others, and counterfeit others,” the CAC said. 

The EU also has rules in place restricting deepfakes online, including fining companies who do not have policies in place to detect and remove disinformation and bot accounts. In 2019, Google made a library of thousands of AI-manipulated videos publicly accessible, hoping that researchers would use the material to develop tools to better detect deceitful content.

Over the last few years, China has been quick to regulate technologies seen as potential threats to the country's social and political stability. 

As a result, several homegrown tech giants have been forced to hand over details about their algorithms to authorities, and video game companies have had to introduce a cap on the amount of time children could spend playing games. 

One of the most visible examples of the crackdown took place in 2020, when the authorities blocked Ant Group's IPO just days after its founder Jack Ma criticised local regulators.

In November last year, E&T interviewed author Michael Grothaus about his book ‘Trust No One’, in which he argues that AI-enabled deepfakes have moved on from amusing digital manipulation to something much more sinister.