# AI firms back shared publisher opt-out signal for web scraping

- URL: https://talkfuse.com/story/the-biggest-names-in-ai-are-backing-a-new-way-to-opt-out-of-scraping/
- Published: 2026-10-30T20:26:10+00:00
- Updated: 2026-10-30T20:47:45+00:00
- Original publisher: The Verge
- Original article: https://www.theverge.com/tech/1023763/openai-google-microsoft-ai-scraping-standard-opt-out
- Topics: AI, Cricket, Government, Politics, Sport, Technology

## Key points

- Four major AI firms back a shared scraping opt-out signal.
- Publishers would add the preference to robots.txt.
- Rollout timing varies, with support planned from 2025.

## Summary

OpenAI, Google, Microsoft and Anthropic say they will support a shared signal allowing publishers to opt out of AI companies scraping their websites. The proposal is intended to let publishers set their preferences once, rather than configuring them separately for each company, as AI crawlers continue to collect online content for model development.

The signal, proposed by Anthropic standards lead Kevin Kelley through the Internet Engineering Task Force, would be added to a site’s robots.txt file. Anthropic has already adopted it; Google and Microsoft plan to support it next year, while OpenAI says it will do so in the coming months. Microsoft will also let publishers distinguish between AI training and AI grounding, while keeping sites in Bing search results.

---
Summarised by TalkFuse (https://talkfuse.com/). Quote the original publisher for the reporting and link this page for the summary and discussion.