Hugging Face Platform Found Hosting Widespread Nonconsensual Deepfake Nude Tools
A study by AI Forensics reveals that most image-editing Spaces on Hugging Face can easily generate nonconsensual nude images, with the majority of user prompts seeking sexualized alterations, often targeting women and sometimes children.
AI Forensics examined nine of Hugging Face's most used image-editing Spaces and discovered that seven could readily produce topless versions of clothed women using a simple prompt. In a separate honey-pot test, the group logged more than 1,000 prompts over a week, finding that 73% were sexual in nature, with 83% seeking to undress the uploaded person and 95% of those requests involving women; 6.7% appeared to target children.
The researchers observed that many Spaces do not identify themselves as nudifying tools, instead presenting as general image editors, and that the platform's safety mechanisms are largely absent at the platform level. Additional review identified multiple pages advertising nudifying technologies and models capable of generating sexualized images of public figures. Hugging Face declined to comment on its moderation processes, though it maintains policies prohibiting child sexual abuse material and non-consensual deepfake content. The findings highlight a gap between policy and enforcement on a major open-source AI repository, raising concerns about the ease with which harmful sexual deepfakes can be created and distributed.
Why it matters
The ease of creating nonconsensual sexual deepfakes on a major AI platform threatens privacy and can enable harassment worldwide.
How the sides frame it
HIGH AGREEMENTBoth camps criticize Hugging Face for hosting tools that create non-consensual sexual deepfakes and note the platform’s weak safety measures, with no substantive disagreement.
LEFT
Exposes widespread non-consensual deepfake nude tools on Hugging Face and highlights the platform’s largely absent safety mechanisms.
CENTER
Reports that Hugging Face is plagued by adult deepfake models and lax safeguards, stressing its contribution to the risk of non-consensual intimate imagery.
The left emphasises
- seven of nine examined Spaces could readily undress women with a simple prompt
- 73% of honey-pot prompts were sexual, 6.7% appeared to target children
- no safeguards are being implemented at a platform level
In this story
Related stories
2 in this thread