--- title: "AI agents and publishing | Am I Ready for Agents?" description: "What AI agents do in publishing today, with 5 dated examples. Three decisions for leadership and the asks to send your web team. Sourced, reviewed 4 Nov 2026." canonical: https://amireadyforagents.com/for/publishers site: "Am I Ready for Agents?" last_updated: 2026-11-05 section: "Sectors" --- # AI agents and publishing | Am I Ready for Agents? > What AI agents do in publishing today, with 5 dated examples. Three decisions for leadership and the asks to send your web team. Sourced, reviewed 4 Nov 2026. Source: https://amireadyforagents.com/for/publishers Site: Am I Ready for Agents? Last reviewed: 2026-11-05 # What AI agents mean for publishers and media ## In short Publishers face agent traffic that grew more than 1,700 percent in a year on Cloudflare's network, and a new set of controls that separate training, search and live assistant fetches. This page lists what changed in 2026, the three decisions that are yours and the asks for the web team. ## What agents do in publishing today Written for the publisher, the head of audience and the commercial lead who decide what AI systems may do with the articles, and for the web lead who sets the CDN rules. - 30 Sep 2026 Cloudflare said on 30 September 2026 that daily requests from AI agents on its network grew by more than 1,700 percent over the past year, and that an agent comes back each time someone asks about a piece of content. [Cloudflare: The Internet has a second audience (30 Sep 2026)](https://blog.cloudflare.com/agentic-web/), read 4 Nov 2026 - 15 Sep 2026 Cloudflare shipped a Disallow AI Training setting on 15 September 2026 that lets sites stay indexed for search by Apple, Google and Microsoft crawlers while refusing training. [Cloudflare: Disallow AI Training, Accountable crawlers (15 Sep 2026)](https://blog.cloudflare.com/accountable-mixed-use-ai-crawlers/), read 4 Nov 2026 - 21 Aug 2026 Cloudflare launched Bot Preference Sync on 21 August 2026, which writes the AI bot choices set for Search, Agent and Training into the site's robots.txt. [Cloudflare: Bot Preference Sync (21 Aug 2026)](https://blog.cloudflare.com/bot-preference-sync/), read 4 Nov 2026 - 14 Aug 2026 Ora's report of 8 Oct 2026, citing Promptwatch, says Reddit's share of ChatGPT search citations fell from 3.83 percent to 0.52 percent in August 2026 as ChatGPT began running site: searches against official sites. Vendor claim. [Ora research: AX is the new AEO (8 Oct 2026)](https://ora.ai/blog/ax-is-the-new-aeo?utm_source=amireadyforagents.com&utm_medium=referral&utm_campaign=sector), read 4 Nov 2026 - 1 Jul 2026 From 15 September 2026, new domains on Cloudflare block Training and Agent bots on pages that display ads by default, while Search stays allowed. [Cloudflare changelog: New options to manage AI traffic (1 Jul 2026)](https://developers.cloudflare.com/changelog/post/2026-07-01-ai-traffic-options/), read 4 Nov 2026 ## Decisions for leadership These three belong to the editorial and commercial lead, not to the developers. Each can be settled in a meeting, and each changes what the web team is asked to build. 1. Whether we allow AI training, AI search indexing and live assistant fetches, decided as three separate choices. 2. Whether to charge AI crawlers or agents, and what loss of reach we accept to test it. 3. Whether agents may fetch pages that carry ads, given the new Cloudflare defaults. - [Which AI bots and agents to let in](https://amireadyforagents.com/decisions/which-bots-to-allow)Four kinds of AI visitor, four separate choices, one robots.txt that matches your bot protection. - [Whether to charge AI crawlers and agents](https://amireadyforagents.com/decisions/whether-to-charge-crawlers)Revenue against reach: what charging for access can cost you in AI answers. - [How to tell verified agents from unknown bots](https://amireadyforagents.com/decisions/verifying-agents)Whether agents that prove who runs them are treated differently from unknown bots. ## The numbers that frame the risk Figures labelled as vendor claims come from Ora or another vendor's own publication and are given with the date they were read. The crawl figures are this site's own. - 5 Nov 2026 On Ora's public leaderboard read 5 Nov 2026, the 36 entries in the News & Intelligence category averaged 48.6 out of 100; 1 scored B (70) or above and 0 scored A (86) or above. Vendor claim. [Ora public leaderboard, category averages](https://ora.ai/leaderboard?utm_source=amireadyforagents.com&utm_medium=referral&utm_campaign=sector), read 5 Nov 2026 - 4 Nov 2026 In the network's own crawl, run of 4 Nov 2026 (13,071 of 13,071 domains from Tranco list 746WX plus Ora leaderboard domains), 14.9 percent of 8,624 sites with a readable robots.txt disallowed CCBot and 13.9 percent disallowed GPTBot at the root; 7.4 percent published a Content-Signals line. [Scorecard methodology: the crawl of 4 Nov 2026](https://amireadyforagents.com/scorecards/methodology), read 4 Nov 2026 - 30 Sep 2026 Cloudflare reported on 30 September 2026 that fewer than 1 percent of sites on its network block search crawlers, while 17 percent block training. [Cloudflare: The Internet has a second audience (30 Sep 2026)](https://blog.cloudflare.com/agentic-web/), read 4 Nov 2026 - 30 Sep 2026 Cloudflare reported that 52 percent of crawler requests it saw in June 2026 were for AI training by stated purpose, up from 22 percent in spring 2025. [Cloudflare: The Internet has a second audience (30 Sep 2026)](https://blog.cloudflare.com/agentic-web/), read 4 Nov 2026 ## Where publishing stands on Ora's leaderboard Ora ranks sites on a public leaderboard, grouped by category. On 5 November 2026 it listed 110 sites in the Gaming & Media and News & Intelligence categories, the ones that stand in for publishing in this site's crawl. The median score was 47 out of 100 and the top score was 89. 4 of the 110 scored B (70) or above. These are Ora's figures, vendor-published, and they describe the sites Ora lists, not every business in the sector. **Your sector on Ora's leaderboard: scores of the 110 listed publishing sites.** Number of sites per score band: 27 in 30 to 39, 40 in 40 to 49, 23 in 50 to 59, 16 in 60 to 69, 2 in 70 to 79, 2 in 80 to 89. The median is 47. A B starts at 70 and an A at 86. Ora's leaderboard, 5 Nov 2026. | Site | Ora category | Score | Grade | Scanned | | --- | --- | --- | --- | --- | | magichour.ai | Gaming & Media | 89 | A | 1 Nov 2026 | | beehiiv.com | Gaming & Media | 88 | A | 1 Nov 2026 | | ZoomInfo | News & Intelligence | 78 | B | 1 Nov 2026 | The middle half of the listed sites scored between about 40 and 56. A score of 40 or lower sits in the bottom quarter of the publishing sites Ora lists, and 56 or higher in the top quarter. To place your own site, scan it with [Ora](https://ora.ai/?utm_source=amireadyforagents.com&utm_medium=referral&utm_campaign=sector) (free, public pages only) and read the result against these numbers, then see [how competitors in your sector score](https://amireadyforagents.com/how-competitors-in-your-sector-score). ## Where publishing stands in the crawl - 61%allow the four main AI agents 119 of 194 sites with a readable robots.txt let GPTBot, ClaudeBot, PerplexityBot and Google-Extended in at the root. - 86%show readable text on the home page 144 of 167 home pages gave a plain fetch 150 words or more of text. The sample is 227 publishing domains from the crawl of 4 Nov 2026. The sector figures use Ora leaderboard categories as stand-ins, not a definition of the sector: Gaming & Media (74 rows), News & Intelligence (36 rows), plus 117 domains placed by a keyword on the domain name. The [full publishing scorecard](https://amireadyforagents.com/scorecards/publishers) has all four measures and the method. ## Check your own business - [Self-check for publishing](https://amireadyforagents.com/self-check/publishers)The 18 core questions with examples from this sector, plus two extra questions. Scored in your browser. - [Boardroom brief for publishing](https://amireadyforagents.com/brief/publishers)One page you can paste into a deck, with the decisions and the numbers above. - [Scorecard for publishing](https://amireadyforagents.com/scorecards/publishers)The four crawl measures for this sector and how they were counted. ## Sources 1. [Cloudflare: The Internet has a second audience (30 Sep 2026)](https://blog.cloudflare.com/agentic-web/) (blog.cloudflare.com/agentic-web), read 4 Nov 2026. 2. [Cloudflare: Disallow AI Training, Accountable crawlers (15 Sep 2026)](https://blog.cloudflare.com/accountable-mixed-use-ai-crawlers/) (blog.cloudflare.com/accountable-mixed-use-ai-crawlers), read 4 Nov 2026. 3. [Cloudflare: Bot Preference Sync (21 Aug 2026)](https://blog.cloudflare.com/bot-preference-sync/) (blog.cloudflare.com/bot-preference-sync), read 4 Nov 2026. 4. [Ora research: AX is the new AEO (8 Oct 2026)](https://ora.ai/blog/ax-is-the-new-aeo?utm_source=amireadyforagents.com&utm_medium=referral&utm_campaign=sector) (ora.ai/blog/ax-is-the-new-aeo), read 4 Nov 2026. Vendor claim. 5. [Cloudflare changelog: New options to manage AI traffic (1 Jul 2026)](https://developers.cloudflare.com/changelog/post/2026-07-01-ai-traffic-options/) (developers.cloudflare.com/changelog/post/2026-07-01-ai-traffic-options), read 4 Nov 2026. 6. [Ora public leaderboard, category averages](https://ora.ai/leaderboard?utm_source=amireadyforagents.com&utm_medium=referral&utm_campaign=sector) (ora.ai/leaderboard), read 5 Nov 2026. Vendor claim. 7. [Scorecard methodology: the crawl of 4 Nov 2026](https://amireadyforagents.com/scorecards/methodology) (amireadyforagents.com/scorecards/methodology), read 4 Nov 2026. 8. [Ora leaderboard API (GET /api/leaderboard)](https://ora.ai/api/leaderboard?utm_source=amireadyforagents.com&utm_medium=referral&utm_campaign=sector) (ora.ai/api/leaderboard), read 5 Nov 2026. Vendor claim. 9. [Sector scorecards dataset, edition 4 Nov 2026](https://amireadyforagents.com/scorecards) (amireadyforagents.com/scorecards), read 4 Nov 2026. Last reviewed 5 November 2026 Reviewed 5 November 2026. [9 sources](#sources), each with the day it was read. ## Take this to your web team 1. Show us our robots.txt and CDN settings for Search, Agent and Training traffic side by side and flag any mismatch. 2. Report monthly AI crawler and agent requests by operator from our CDN logs. 3. Confirm article text is in the HTML for readers we allow, with paywall rules stated. 110 publishing sites sit on Ora's leaderboard with a median score of 47, and magichour.ai leads at 89. [Scan yours with Ora](https://ora.ai/?utm_source=amireadyforagents.com&utm_medium=referral&utm_campaign=sector) to see which side of 47 it falls. Limits: public pages only, no published data on repeat scans, and a waitlist for its fixing service. [Which tool should our team use](https://amireadyforagents.com/which-agent-readiness-tool)