# Cloudflare to block mixed-use AI crawlers on ad pages by default from Sept 15 unless separated by purpose

_Monday, July 6, 2026 at 8:00 AM EDT · Power · Latest · Tier 2 — Notable_

![Cloudflare to block mixed-use AI crawlers on ad pages by default from Sept 15 unless separated by purpose — Primary](https://techcrunch.com/wp-content/uploads/2022/09/Matthew-Prince-co-founder-CEO-Cloudflare-1.jpg?resize=1200,800)

Cloudflare announced it will begin blocking mixed-use AI web crawlers by default on ad-supported websites starting September 15, 2026. The company is giving AI providers until that date to separate their web crawlers into distinct categories for search indexing, AI agent activity, and AI model training. Under the new policy, crawlers that combine multiple purposes without clearly identifying them will be subject to the most restrictive access setting. Training and agent crawlers will be blocked from ad-supported pages by default, while search crawlers will remain allowed unless site owners choose otherwise. The defaults apply to new customers, new sites added by existing customers, and existing free-tier customers. Website owners can opt out of the changes through their Cloudflare dashboard settings before the deadline. Mixed-use crawlers such as those from major providers will be blocked on ad pages if they do not separate their functions.

## Sources

- [TechCrunch](https://techcrunch.com/2026/07/01/cloudflares-new-policy-pushes-ai-companies-to-pay-for-publishers-content/)

---
Canonical: https://techandbusiness.org/newswire/aD45H1NEbb1bqELwluWf4i
Retrieved: 2026-07-06T22:49:00.024Z
Publisher: Tech & Business (techandbusiness.org)