Cloudflare adds AI-training control for mixed-use crawlers

Image: Primary

Published Tuesday, September 15, 2026 · 10:07 AM EDT

Cloudflare introduced a Disallow AI Training setting that lets site owners keep qualifying mixed-use crawlers available for search while expressing a no-training preference through robots.txt. Cloudflare says Apple, Google and Microsoft honor, or have committed on specified timelines to honor, the setting. The change also makes Cloudflare's Block controls apply to mixed-use crawlers, meaning those controls can affect search indexing. Existing customer settings will be migrated to the new controls.

Sources

The Cloudflare Blog

In this story

Google Microsoft Apple Cloudflare

Published by Tech & Business, a media brand covering technology and business. This story was sourced from The Cloudflare Blog and reviewed by the T&B editorial agent team.

Back to Newswire

Keep reading

Full wire

AI Products

Amazon sets up supplier financing to borrow as much as $80 billion

Amazon has established supplier financing arrangements to borrow as much as $80 billion against the value of hardware it buys, Bloomberg reported.

AI Infrastructure

Five technology giants commit more than $3T to future spending

Alphabet, Meta, Microsoft, Amazon and Oracle have committed more than $3T to future spending on data centres, leases, equipment and energy, The Next Web reported, citing Bloomberg. Alphabet accounts for more than $1T and Meta for more than $800B.

Security Infrastructure

Linux flaw enables remote root exploit on servers using Multipath TCP

Anthropic researchers demonstrated remote root command execution through CVE-2026-98432 on a Linux server running a Multipath TCP listener, BleepingComputer reported.

Science

Researchers demonstrate faster water transport with more selective membrane

University of Queensland researchers demonstrated a membrane that increased water transport by approximately 44% and reduced ethanol transport by 90% in ethanol-water separation experiments, Phys.org reported.

AI Policy

AI election PAC spending reaches nearly $80M, with regulatory advocates ahead

AI-oriented political action committees spent nearly $80M on the 2026 US elections, mostly before the primaries, Puck reported, citing FEC data.

AI Science

Insilico says AI-discovered Rentosertib has entered Phase III trials

Insilico Medicine says Rentosertib, its generative AI-discovered drug candidate also known as ISM001-055, has entered Phase III clinical trials for idiopathic pulmonary fibrosis. The company described the clinical advance in an announcement carried by News-Medical.Net.