OpenAI introduces text watermark for EU ChatGPT and Codex output
Developing story first seen 16 hours ago
OpenAI has launched textGrain, a watermarking system for AI-generated text designed to comply with the EU's AI Act requirement that model output be machine-readable. The technology will be mandatory for ChatGPT and Codex output in the European Union within weeks, though API customers can choose to opt in globally. This is part of a broader regulatory push to make AI-generated content identifiable to combat misinformation and track content provenance.
The watermark works by subtly altering word choices based on statistical patterns, but it has significant practical limitations. Detection rates vary considerably: the system catches approximately 80 per cent of watermarks in 200-word passages, improving to 95 per cent in longer 400-word texts, and performs poorly with functional writing like code or mathematics. Critically, the watermark is easily defeated: substituting just 10 per cent of words with synonyms reduces detection from 92 per cent to 66 per cent, and replacing 25 per cent of words allows the watermark to go undetected in 83 per cent of cases. Unlike rival Anthropic, which applies watermarking globally, OpenAI is restricting mandatory implementation to Europe.
- The watermark only catches 80 per cent of AI text in short passages
- Word substitution easily defeats the watermarking system
- OpenAI limits mandatory watermarking to Europe, unlike Anthropic
New here? Start with this
The European Union has introduced rules requiring artificial intelligence companies to mark their machine-generated text in ways that can be read by both humans and software. This stems from concern about misinformation and the need to track where content comes from as AI-generated text becomes more common. Regulators want people to be able to tell AI writing from human writing.
Watermarking is a technique that invisibly marks content to identify its source or nature. OpenAI, the company behind ChatGPT, has developed a watermarking system to comply with the EU requirement. The technology works by subtly altering the way the AI chooses words when generating text.
As AI tools become widely used for writing, the ability to identify machine-generated content has become important for journalists, educators, and others who need to verify information. The introduction of these markers is part of a wider effort to ensure transparency about when AI has been used. Different AI companies are taking varying approaches to implementation across different regions.
Both sides, in good faith
The strongest fair case each way — we don't pick a winner.
The case for
Advocates of the watermarking system argue that transparency about AI-generated content is essential for combating misinformation, maintaining public trust, and ensuring responsible technology deployment. They contend that even imperfect detection mechanisms create accountability and establish a clear regulatory standard that pushes the AI industry toward greater transparency. The values underlying this position emphasise public interest protection, regulatory responsibility, and the principle that content provenance should be traceable.
The case against
Critics contend that the watermarking system's significant technical limitations render it inadequate for its stated purpose. They point out that detection rates degrading from 92 to 66 per cent with merely 10 per cent word substitution, combined with poor performance on code and mathematical content, means the watermark provides false reassurance rather than genuine protection against misuse. From this perspective, resources would be better invested in more robust approaches like cryptographic authentication and digital signatures, and the fragmented EU-only mandate creates inconsistent standards without delivering meaningful security.
More coverage
Read the full article at the source →
Originally published by The Register as “OpenAI rolls out weak sauce watermarking for AI text”.