A study by European non-profit AI Forensics has found that Hugging Face, the leading open-source AI hosting platform, is hosting numerous tools capable of generating non-consensual sexual deepfakes with little to no moderation. The findings come as the European Union and the UK move to ban "nudification" apps, and they highlight how open platforms designed for sharing and benchmarking AI models can be exploited to create intimate imagery without subjects' consent.

Researchers tested the nine most popular image-editing "Spaces" on the platform using an AI-generated photo of a woman and a prompt asking for a topless version, and seven of the nine tools complied. When AI Forensics deployed its own dummy image-editing model without generating actual outputs, it collected 1,081 user prompts within a week, 73% of which were sexual in nature and 83% seeking to undress the subject; 95% of targeted subjects were women, and 6.7% of sexual requests involved a minor. The group found that only 3% of audited Spaces had any output moderation, despite Hugging Face's policies explicitly prohibiting non-consensual sexual imagery.