Technology news and discussion on TalkFuse.
Hugging Face reportedly plagued with AI models generating adult deepfakes
A study by European non-profit AI Forensics has found that Hugging Face, the leading open-source AI hosting platform, is hosting numerous tools capable of generating non-consensual sexual deepfakes with little to no moderation. The findings come as the European Union and the UK move to ban "nudification" apps, and they highlight how open platforms designed for sharing and benchmarking AI models can be exploited to create intimate imagery without subjects' consent.
Researchers tested the nine most popular image-editing "Spaces" on the platform using an AI-generated photo of a woman and a prompt asking for a topless version, and seven of the nine tools complied. When AI Forensics deployed its own dummy image-editing model without generating actual outputs, it collected 1,081 user prompts within a week, 73% of which were sexual in nature and 83% seeking to undress the subject; 95% of targeted subjects were women, and 6.7% of sexual requests involved a minor. The group found that only 3% of audited Spaces had any output moderation, despite Hugging Face's policies explicitly prohibiting non-consensual sexual imagery.
In short
- Hugging Face hosts AI tools that create non-consensual sexual deepfakes
- Study found 7 of 9 top tools would generate topless images on request
- Only 3% of tested Spaces had any output moderation in place
Explained simply
Hugging Face is an open-source platform where researchers and developers worldwide can host and test artificial intelligence models. Researchers at a European non-profit called AI Forensics have found that many of the image-editing tools on the platform can be misused to create deepfakes—realistic fake images that appear to show real people in scenarios they never consented to.
The concern is non-consensual sexual imagery. The AI tools are being used to create fake intimate pictures of people without permission, particularly by digitally removing clothing or creating explicit scenes, and women are disproportionately targeted. This causes serious harm to victims, and regulatory bodies in the European Union and the United Kingdom are working to prohibit such deepfake tools entirely.
Hugging Face has policies explicitly prohibiting non-consensual sexual imagery, yet enforcement across an open-source platform can be difficult. The issue underscores how tools designed for legitimate AI research can be misused despite stated safeguards.
AI Research Science Technology