---
title: "Pay per crawl: plain-language definition | Am I Ready for Agents?"
description: "Pay per crawl: Charging AI crawlers a fee for each page they fetch, collected through your CDN. What it is, a dated example and where it matters."
canonical: https://amireadyforagents.com/glossary/pay-per-crawl
site: "Am I Ready for Agents?"
last_updated: 2026-11-04
section: "Glossary"
---

# Pay per crawl: plain-language definition | Am I Ready for Agents?

> Pay per crawl: Charging AI crawlers a fee for each page they fetch, collected through your CDN. What it is, a dated example and where it matters.

Source: https://amireadyforagents.com/glossary/pay-per-crawl
Site: Am I Ready for Agents?
Last reviewed: 2026-11-04

# Pay per crawl

## In short

Pay per crawl is the option behind charging AI crawlers, a trade of revenue against reach. Whether to charge AI crawlers and agents goes into the detail. The neighbouring terms are AI crawler and x402. Cloudflare's AI Crawl Control documentation includes setting a pay per crawl price (read 4 Nov 2026).

## Definition

Charging AI crawlers a fee for each page they fetch, collected through your CDN.

## A dated example

-   Read 4 Nov 2026
    
    Cloudflare's AI Crawl Control documentation includes setting a pay per crawl price (read 4 Nov 2026).
    
    [Cloudflare AI Crawl Control docs](https://developers.cloudflare.com/ai-crawl-control/), read 4 Nov 2026
    

## Where it matters

It is the option behind charging AI crawlers, a trade of revenue against reach.

-   [Whether to charge AI crawlers and agents](https://amireadyforagents.com/decisions/whether-to-charge-crawlers)

## Related terms

-   [AI crawler](https://amireadyforagents.com/glossary/ai-crawler)A bot that collects pages in bulk on its own schedule, to train a model or build an AI search index.
-   [x402](https://amireadyforagents.com/glossary/x402)A way for a site to ask for payment inside the web request itself, so an agent can pay a small amount and retry without an account.

## Sources

1.  [Cloudflare AI Crawl Control docs](https://developers.cloudflare.com/ai-crawl-control/) (developers.cloudflare.com/ai-crawl-control), read 4 Nov 2026.

Last reviewed 4 November 2026

Reviewed 4 November 2026. [1 source](#sources), each with the day it was read.