Technology news and discussion on TalkFuse.
OpenAI releases a new image classifier and ChatGPT warnings to combat CSAM
OpenAI has announced new measures to detect child sexual abuse material (CSAM) and discourage users from creating or accessing content that sexualises children. The changes include an image classifier, new warnings and temporary account blocks in ChatGPT, developer guidance and AI tools for law enforcement.
The classifier can assess previously unseen images, including AI-generated ones, and give reviewers an indicative score. In early partner testing, it flagged 48% fewer images while identifying the same number of abusive images; those it flagged were twice as likely to be abusive. ChatGPT will show a warning and direct users seeking harmful prompts to specialist support, with a 24-hour suspension for a first incident and possible permanent removal for repeated offences.
In short
- OpenAI released a classifier to help identify CSAM, including AI-generated images.
- Early testing found it flagged 48% fewer images while detecting the same number of abusive ones.
- ChatGPT will warn users and temporarily suspend accounts for harmful prompts.
Steelman: both sides, in good faith
The strongest fair case each way — we don't pick a winner.
The case for
These measures could help identify abusive material, including newly generated images, and direct users towards support before harmful behaviour escalates. Warnings and temporary suspensions may deter misuse, while tools for reviewers and law enforcement could improve the response to a serious harm affecting children.
The case against
Even well-intended classifiers can make mistakes, and account suspensions based on automated detection may unfairly restrict users if decisions are not carefully reviewed and appealable. Expanding detection and law-enforcement tools also raises questions about privacy, oversight and how reliably indicative scores distinguish abuse from lawful material.
Explained simply
Child sexual abuse material (CSAM) refers to images or other content that depict or sexualise children. It is illegal in many countries to create, possess or share such material, and online services have systems for finding and reporting it.
OpenAI makes ChatGPT and other artificial intelligence tools. Its services can generate or process images and text, so the company’s safeguards are intended to identify harmful material and discourage its creation or access; specialist organisations and law enforcement also work to protect children and investigate abuse.