Corroborated event · 4 outlets
Google, OpenAI, Microsoft and Anthropic said they will adopt a new common robots.txt signal that lets website operators block scraping for AI training.
AI brief
Google, Microsoft, Anthropic and OpenAI said they plan to support an upcoming website crawling standard that would let site operators block AI developers from using their content for training.
Why it matters: It would give publishers a formal way to opt out of AI training scrapes.
Written by AI from SiliconANGLE AI's published text.
Background
From our own funding, model and news trackers, not from the source.
OpenAI
- Latest funding
- $30B · 31 Oct 2026
- Newest model on the API
- GPT-6.1 Terra Pro · 26 Oct 2026 · $1 in / $5 out per million tokens
Anthropic
- Latest funding
- Growth · $65B · 28 May 2026
- Newest model on the API
- Claude Haiku 5.5 · 7 Oct 2026 · $0.1 in / $0.5 out per million tokens
More coverage of this story (3)
Google, OpenAI, Microsoft and Anthropic back new scraping opt-out standard
Google joins OpenAI and Anthropic in promising new scraping controls
Engadget AI · 30 Oct 2026, 22:45
The biggest names in AI are backing a new way to opt out of scraping
The Verge AI · 30 Oct 2026, 20:26
Making it easier for websites to express AI training preferences
OpenAI News · 30 Oct 2026, 09:30
Primary source
“Google LLC, Microsoft Corp., Anthropic PBC and OpenAI Group PBC today announced plans to support an upcoming website crawling standard.”
Reporting from a publisher outside the organisation being covered.
Read original story ↗