San Francisco Demands Apple and Google Delete AI ‘Nudify’ Apps From App Stores
San Francisco’s city attorney, David Chiu, has sent cease-and-desist letters to Apple and Google demanding the removal of 13 AI face-swapping apps that can generate non-consensual nude images. The action matters because the city argues the companies may be profiting from technology that facilitates sexual abuse and that stronger app-store moderation is needed to prevent such tools reaching users.
The letters cite California laws against supporting services that create deepfake pornography and say the platforms take a share of in-app payments. Google said it had removed hundreds of violating apps, including the five Android apps identified by the city, while Apple said it had removed three flagged apps and was terminating their developer accounts; other developers were told to address policy breaches or risk removal. Both firms prohibit pornography, abuse and harassment under their developer policies, but researchers have repeatedly found nudification apps in their stores, sometimes with child-suitable ratings.
In short
- San Francisco demands Apple and Google remove 13 nudification apps.
- The city says app stores may be profiting from illegal deepfake pornography.
- Google and Apple say they have removed several offending apps.
Steelman: both sides, in good faith
The strongest fair case each way — we don't pick a winner.
The case for
These apps explicitly enable sexual abuse by creating non-consensual intimate imagery, directly violating Apple and Google's policies against exploitation and harassment. The platforms control the distribution channels and profit from in-app purchases generated by these tools, placing responsibility squarely on them. Meaningful protection requires swift removal to prevent further harm to identifiable victims; regulatory and law enforcement processes move too slowly to protect people being abused in real time.
The case against
Whilst non-consensual intimate imagery represents unconscionable abuse, requiring tech platforms to serve as content enforcers is problematic. Criminal responsibility properly belongs to those who create and distribute abusive material; this should be addressed through vigorous law enforcement prosecution under existing statutes. App store removals offer incomplete solutions, as such tools will simply migrate to alternative distribution channels. Establishing expectations of comprehensive platform moderation risks suppressing legitimate content, setting dangerous precedent, and diverting resources from where they're needed most: criminal enforcement against perpetrators.
Explained simply
Artificial intelligence tools can now create fake nude images of real people by placing their faces onto explicit photographs. These "nudification" apps create non-consensual sexual imagery that can be used to harass, blackmail or humiliate victims without their knowledge. Such tools have been flagged by campaigners as a form of image-based sexual abuse.
Apple and Google operate the world's two largest mobile app stores and effectively control what software reaches most smartphone users. Both companies have stated policies against pornography, sexual abuse and harassment, meaning such apps should be prohibited. Yet researchers have repeatedly found nudification apps available for download, sometimes with age ratings suggesting they are appropriate for children.
California has specific laws designed to prevent services from facilitating the creation of non-consensual intimate imagery. Questions remain about whether app-store operators should take primary responsibility for preventing such tools being distributed on their platforms in the first place. This issue of content moderation and corporate accountability has become increasingly contentious in recent years.