What Cloudflare's new AI defaults mean for you
In short
Cloudflare changed what AI bot controls exist and what is on by default in 2026. Since 1 July every plan can block Search, Agent and Training traffic separately. From 15 September new domains block Training and Agent bots on pages with ads, and a Disallow AI Training setting keeps search while refusing training. Check your own settings.
What to check
- Which zone settings exist for Search, Agent and Training traffic, and whether each is allowed, blocked on pages with ads or blocked everywhere.
- Whether the domain was added after 15 September 2026. If so, the new defaults may be on and nobody chose them.
- Whether Bot Preference Sync is writing your choices into robots.txt, so the stated preference and the enforced rule match.
- Whether the Disallow AI Training setting matches the position you decided in which AI bots to let in.
If your site runs elsewhere, ask your host or CDN the same four questions in its own words.
What the evidence says
Explain the July and September 2026 Cloudflare changes to a leader whose site runs on Cloudflare.
- 30 Sep 2026
Cloudflare reported fewer than 1 percent of its sites block search crawlers and 17 percent block training.
- 15 Sep 2026
Cloudflare's Disallow AI Training setting, launched 15 September 2026, keeps Apple, Google and Microsoft search crawling while refusing training.
- 21 Aug 2026
Cloudflare's Bot Preference Sync, launched 21 August 2026, writes those choices into robots.txt.
- 1 Jul 2026
From 15 September 2026, new domains on Cloudflare block Training and Agent bots on pages with ads by default, while Search stays allowed.
- 1 Jul 2026
Cloudflare groups AI traffic into Search, Agent and Training, each blockable on all pages, on ad pages only, or not at all, on every plan since 1 July 2026.
Related
- Which AI bots to let inFour kinds of visitor, four choices.
- Ask your web teamThe sentences to send, grouped by area.
- The self-checkEighteen questions scored in your browser.
Sources
- Cloudflare: The Internet has a second audience (30 Sep 2026) (blog.cloudflare.com/agentic-web), read 4 Nov 2026.
- Cloudflare: Disallow AI Training, Accountable crawlers (15 Sep 2026) (blog.cloudflare.com/accountable-mixed-use-ai-crawlers), read 4 Nov 2026.
- Cloudflare: Bot Preference Sync (21 Aug 2026) (blog.cloudflare.com/bot-preference-sync), read 4 Nov 2026.
- Cloudflare changelog: New options to manage AI traffic (1 Jul 2026) (developers.cloudflare.com/changelog/post/2026-07-01-ai-traffic-options), read 4 Nov 2026.
Last reviewed