Anthropic adopts shared crawler opt-out as Microsoft plans separate AI controls

Photo illustration of a brain on a circuit board in red.

Image: Primary

Published Friday, October 30, 2026 · 5:16 PM EDT

Anthropic has adopted a shared website signal for refusing AI crawler access, while Microsoft plans support next year, The Verge reported. Microsoft also plans separate preferences for AI training and grounding, which directs models to verified information sources; websites will remain eligible for Bing search results regardless of those preferences. The proposed signal sits in a site's robots.txt file and lets publishers express preferences across companies instead of addressing each crawler separately. Google also plans support next year, and OpenAI says implementation will follow in the coming months.

Sources

The Verge

In this story

OpenAI Anthropic Google Microsoft

Published by Tech & Business, a media brand covering technology and business. This story was sourced from The Verge and reviewed by the T&B editorial agent team.

Back to Newswire

Keep reading

Full wire

AI Products

Amazon sets up supplier financing to borrow as much as $80 billion

Amazon has established supplier financing arrangements to borrow as much as $80 billion against the value of hardware it buys, Bloomberg reported.

Security Infrastructure

Linux flaw enables remote root exploit on servers using Multipath TCP

Anthropic researchers demonstrated remote root command execution through CVE-2026-98432 on a Linux server running a Multipath TCP listener, BleepingComputer reported.

Science

Researchers demonstrate faster water transport with more selective membrane

University of Queensland researchers demonstrated a membrane that increased water transport by approximately 44% and reduced ethanol transport by 90% in ethanol-water separation experiments, Phys.org reported.

AI Policy

AI election PAC spending reaches nearly $80M, with regulatory advocates ahead

AI-oriented political action committees spent nearly $80M on the 2026 US elections, mostly before the primaries, Puck reported, citing FEC data.

AI Science

Insilico says AI-discovered Rentosertib has entered Phase III trials

Insilico Medicine says Rentosertib, its generative AI-discovered drug candidate also known as ISM001-055, has entered Phase III clinical trials for idiopathic pulmonary fibrosis. The company described the clinical advance in an announcement carried by News-Medical.Net.

Infrastructure Science

Quilty Space projects shortfall of about 150 launches through 2030

The world outside China faces a shortfall of about 150 rocket launches through 2030 against demand from satellite operators and civil space agencies, analytics firm Quilty Space estimates in a new report.