San Francisco Orders Meta to Stop Allowing AI Child

Meta's Ad Policy Failures Exposed

According to empirical research synthesized by Groundwork, Meta's ad policies and safety systems have failed to address a known issue of AI-generated child sexually explicit ads, despite profiting from them.

On Tuesday, WIRED reported that Meta has run more than 350 ads in recent months that transformed still images of minors—some of them confirmed as real individuals, including a member of a European royal family —into short videoclips that can depict them performing sexual acts. When clicked, these ads would direct people to download AI image- and video-generation apps, some of which could be used to digitally undress people or create other nonconsensual intimate imagery.

The Ads in Question

The ads, which were discovered by researchers at the Tech Transparency Project (TTP), included some identical ads that had been uploaded multiple times, and in total they reached more than 29,000 accounts in countries in the European Union. The ads that were shown across Facebook, Instagram, and Threads also targeted accounts in the US, Australia, and India. WIRED initially reported on a batch of 53 of the ads at the start of August, but more than 250 of the total number of ads ran on Meta’s platforms after this initial report.

San Francisco Takes Action

San Francisco city attorney David Chiu says in a statement to WIRED, “Meta has failed to address a known issue of AI-generated child sexually explicit ads, while profiting from them. These ads are deeply disturbing and cause harm to real children. Meta claims to have zero tolerance for child exploitation, yet the scale and persistence of these issues suggest current efforts are woefully inadequate. This is unacceptable, and Meta must do better.”

Meta's Response

Meta says there is “no indication” that the ads were displayed in San Francisco and this means they are “outside the city attorney's jurisdiction.” While Meta’s ad library, which is a repository of ads, includes some data about ads published in the US, it does not break down where in the US ads may have run.

Jurisdictional Claims

In response to Meta's jurisdictional claims, City Attorney's Office spokesperson Alex Barrett-Shorter tells WIRED that, because “San Francisco consumers can access the unlawful ads through Meta online spaces,” the matter is the city of San Francisco's business. “We find Meta’s response astonishing. Meta appears less concerned about whether they’re guilty of exploiting children than whether San Francisco should say anything about it,” Barrett-Shorter says. “Meta has bigger problems to solve here than determining our jurisdiction.”

Meta's Previous Response

Meta previously told WIRED that many of the ads identified by the TTP researchers had already been removed before they were reported—all of them have now been removed for violating its policies—and most had “fewer than 200 impressions” from accounts. “We don’t tolerate nudify apps or any kind of child exploitation, whether real or AI-generated,” a Meta spokesperson previously said.

Related Concerns

This incident raises concerns about the risks of AI-generated content and the need for stricter regulations to prevent the spread of such material. As Groundwork has previously discussed in our analysis on semiconductor capex, the development of AI technologies is rapidly advancing, and it is essential to ensure that these advancements are used responsibly.