---
title: "Scorecard methodology | Am I Ready for Agents?"
description: "How the sector scorecards are built: the crawl, the four measures, how a domain is assigned to a sector, the whole-crawl numbers and the limits."
canonical: https://amireadyforagents.com/scorecards/methodology
site: "Am I Ready for Agents?"
last_updated: 2026-11-05
section: "Scorecards"
---

# Scorecard methodology | Am I Ready for Agents?

> How the sector scorecards are built: the crawl, the four measures, how a domain is assigned to a sector, the whole-crawl numbers and the limits.

Source: https://amireadyforagents.com/scorecards/methodology
Site: Am I Ready for Agents?
Last reviewed: 2026-11-05

# Methodology: how the sector scorecards are built

## In short

The scorecards come from one crawl of 13,071 websites on 4 November 2026, run with a plain client and an honest user agent. Each domain is fetched for robots.txt, the home page and a few discovery files, then counted in a sector and four measures. This page gives every rule and every limit.

## The crawl

-   Universe: the Tranco list 746WX (the top 10,000 domains), plus every domain on Ora's public leaderboard, plus the platforms and vendors named on this site. 13,071 domains in all.
-   Client: a plain HTTP fetch with no JavaScript, the user agent Mozilla/5.0 (compatible; agent-readiness-crawl/1.0) and a ten-second timeout. It honours robots.txt for its own name.
-   Per domain: robots.txt, the home page, the same page asked for as markdown, /llms.txt and /llms-full.txt, and twelve discovery paths such as /.well-known/mcp.json, /openapi.json, /agents.json and /.well-known/ucp.json.
-   Stored: extracted fields only, never page bodies beyond the first 200 words.
-   Status: the whole universe was crawled.

## The four measures

Allow the four main AI agents

Of sites whose robots.txt was readable, the share whose rules allow GPTBot, ClaudeBot, PerplexityBot, Google-Extended at the site root. A group that disallows everything for every visitor counts as blocking all four.

Readable home page

Of home pages that returned 200 to a plain fetch, the share with 150 or more words of text that were not a script-only shell.

Any act surface

Of sites where every path below was probed, the share where any answered 200 with JSON, or YAML for openapi.yaml: /.well-known/mcp.json, /openapi.json, /openapi.yaml, /agents.json, /.well-known/agent.json, /.well-known/agent-card.json.

Any payment file

Of sites where the path below was probed, the share that answered 200 with JSON: /.well-known/ucp.json.

A site that did not answer, or whose robots.txt told our client to stay away, is left out of the measures it could not be counted in. Each percentage shows its own denominator for that reason.

## How a domain gets a sector

Domains from Ora's leaderboard carry a category, and the category decides the sector. Domains from the Tranco list carry none, so a small keyword list on the domain name decides, first match wins, in the order shown.

| Ora leaderboard category | Sector |
| --- | --- |
| E-commerce | Ecommerce and retail |
| Commerce Tools | Ecommerce and retail |
| Travel & Hospitality | Bookings, travel and hospitality |
| News & Intelligence | Publishers and media |
| Gaming & Media | Publishers and media |
| Developer Tools | SaaS and B2B software |
| AI & ML | SaaS and B2B software |
| CRM | SaaS and B2B software |
| Productivity | SaaS and B2B software |
| Communication & Support | SaaS and B2B software |
| Data & Analytics | SaaS and B2B software |
| Legal & Compliance | Professional and B2B services |
| Marketing | Professional and B2B services |
| HR & Recruiting | Professional and B2B services |
| Healthcare | Healthcare and clinics |
| Payments & Fintech | Financial services |
| Consumer | Local and multi-location businesses |
| Real Estate & Construction | Local and multi-location businesses |

| Sector | Keywords |
| --- | --- |
| Healthcare and clinics | health, clinic, hospital, pharma, medic, dental, doctor, therapy |
| Financial services | bank, finance, capital, credit, insurance, loan, invest, wallet, mortgage |
| Bookings, travel and hospitality | travel, hotel, flight, airline, airways, resort, cruise, vacation, holiday, booking, tours |
| Publishers and media | news, times, herald, gazette, tribune, journal, magazine, daily |
| Ecommerce and retail | shop, store, mart, outlet, fashion, buy, deals |
| Professional and B2B services | consult, advisory, legal, lawyer, accounting, attorney |
| SaaS and B2B software | software, cloud, saas, devops |
| Local and multi-location businesses | realty, estate, homes, property, restaurant, cafe, coffee, pizza |

10,026 crawled domains match neither rule and appear only in the all-domains row. Categories not mapped to a sector, such as Infrastructure or Security, are left out of the sector rows.

Of the 13,071 domains crawled, 10,886 were reachable. The eight sector rows hold 3,045 domains, 2,593 placed by category and 452 by keyword. These totals are the all-domains row and the sums of the sector rows in the [scorecards dataset](https://amireadyforagents.com/scorecards).

## Headline numbers from the whole crawl

| Measure | Share | Count |
| --- | --- | --- |
| GPTBot disallowed at the root | 13.9% | 1,198 of 8,624 readable robots.txt files |
| ChatGPT-User disallowed | 6.9% | 595 of 8,624 |
| ClaudeBot disallowed | 13.1% | 1,132 of 8,624 |
| PerplexityBot disallowed | 7.9% | 682 of 8,624 |
| Google-Extended disallowed | 12.1% | 1,040 of 8,624 |
| CCBot disallowed | 14.9% | 1,283 of 8,624 |
| Content-Signal line in robots.txt | 7.4% | 641 of 8,624 |
| Home page answered 401 or 403 to our client | 12.9% | 1,359 of 10,565 home pages fetched |
| Served an llms.txt file | 24.6% | 2,600 of 10,565 |
| Answered a markdown request with markdown | 7.9% | 835 of 10,565 |
| Carried JSON-LD on the home page | 54.0% | 4,743 of 8,783 home pages that returned 200 |
| Home page was a script-only shell | 3.5% | 307 of 8,783 |
| Published any act surface | 2.1% | 216 of 10,385 |
| Published a UCP profile | 0.59% | 62 of 10,439 |

## Ora's figures in the sector guides

The Ora block on each sector guide reads one saved response: a single GET of Ora's leaderboard API on 5 November 2026, which listed 3,453 sites. The response is kept in this site's source with its date, and no figure is typed in. A sector is read through the Ora categories in the table above. The count is the number of listed sites, the median is the middle score (the mean of the two middle scores when the count is even), the middle half runs from the 25th to the 75th percentile by linear interpolation, and the three leaders are the highest scores, with ties broken by name.

Ora's cached scores for the 24 leaders came from GET /api/score for each domain, one request a second. The counts of checks came from GET /api/checks and the totals and layer averages from GET /api/stats, which reported 132,085 ranked companies. Nothing was posted and no scan was started. Every figure is Ora's and vendor-published.

The leaderboard lists a subset of the companies Ora ranks, and its entries were scanned on 123 different days from 7 Apr 2026 to 5 Nov 2026. Read a sector median as a yardstick, not as a census.

## Limits

-   A plain fetch sees what a plain client sees. A site that serves agents something different from what it serves our client is not measured.
-   Our client has a clear name. A site that refuses unknown automation may have refused us, and counts as a refusal here, which is also what a customer's unknown agent would meet.
-   Sector assignment by keyword is rough, and a leaderboard category is Ora's, not ours.
-   Only the paths listed were probed. A site can offer an act surface or a payment route that is not at one of them.
-   One crawl is one day. The next edition will move these numbers.

## The API, the MCP server and the ask endpoint

The scorecards, the self-check questions and the ten decisions are also served to agents, as the [API and MCP server](https://amireadyforagents.com/developers) page documents: an MCP server at `/mcp` with five read-only tools, a REST API under `/api/v1` described by [openapi.json](https://amireadyforagents.com/openapi.json), and an NLWeb `/ask` endpoint that searches the rows and pages and returns schema.org objects with a source URL each.

Every surface reads the published files under `/data/` when it answers, so a tool result and the page it cites come from the same edition, and a count in a result is the count in the CSV. Nothing writes and nothing needs a credential. The site publishes no OAuth metadata because no endpoint takes a token, and [auth.md](https://amireadyforagents.com/auth.md) says so.

## Reproducing a number

1.  Take the sector's domains from the leaderboard category or keyword rule above.
2.  Apply the four conditions to each row of the crawl.
3.  Divide as shown in each row's own count.

The slicing script ships with this site's source, and a test recomputes one sector from the crawl rows and checks it against the published numbers, so a change in either cannot pass unnoticed.

## Sources

1.  [Tranco list 746WX, the crawl universe](https://tranco-list.eu/list/746WX) (tranco-list.eu/list/746WX), read 4 Nov 2026.
2.  [RFC 9309 Robots Exclusion Protocol](https://www.rfc-editor.org/rfc/rfc9309) (rfc-editor.org/rfc/rfc9309), read 4 Nov 2026.
3.  [Ora public leaderboard, category averages](https://ora.ai/leaderboard?utm_source=amireadyforagents.com&utm_medium=referral&utm_campaign=methodology) (ora.ai/leaderboard), read 4 Nov 2026. Vendor claim.
4.  [Ora leaderboard API (GET /api/leaderboard)](https://ora.ai/api/leaderboard?utm_source=amireadyforagents.com&utm_medium=referral&utm_campaign=methodology) (ora.ai/api/leaderboard), read 5 Nov 2026. Vendor claim.
5.  [Ora stats API (GET /api/stats)](https://ora.ai/api/stats?utm_source=amireadyforagents.com&utm_medium=referral&utm_campaign=methodology) (ora.ai/api/stats), read 5 Nov 2026. Vendor claim.
6.  [Ora check catalogue API](https://ora.ai/api/checks?utm_source=amireadyforagents.com&utm_medium=referral&utm_campaign=methodology) (ora.ai/api/checks), read 5 Nov 2026. Vendor claim.

Last reviewed 5 November 2026

Reviewed 5 November 2026. [6 sources](#sources), each with the day it was read.