Reading an agent-readiness scan: which checks gate a level

Who reported it, and when we saw it

Key facts

• The level table is served at https://isitagentready.com/.well-known/agent-skills/scan-site/SKILL.md: 200, text/markdown, 2504 bytes when fetched on 2026-08-09. Cloudflare's 2026-04-17 announcement does not contain the table, but it does give the URL of the skills index that holds it.
• 23 sites scanned through POST https://isitagentready.com/api/scan on 2026-08-09 between 13:48 and 14:44 UTC: 3 returned level 0, 6 level 1, 3 level 2, 1 level 3, 7 level 4, 3 level 5. That is 21 checks per site and 483 check results. The published table predicts all 23 returned levels.
• sentry.io returned level 5 on 2026-08-09 at 14:44 UTC with authMd fail, a2aAgentCard fail, mcpServerCard fail, agentSkills fail, linkHeaders fail and dnsAid fail, passing oauthProtectedResource and oauthDiscovery. developers.cloudflare.com returned level 4 in the same minutes and differs on oauthProtectedResource alone.
• scan-site/SKILL.md documents three check statuses (pass, fail, unableToCheck); across 483 results on 2026-08-09 the API returned four: fail 202, pass 162, neutral 118, unableToCheck 1. webBotAuth was neutral on all 23 sites, never pass and never fail.
• Cloudflare's 2026-04-17 post is 34,061 characters of plain text with zero occurrences of '0-100', '0 to 100', 'out of 100' or '100-point'. The 0-100 scale is ours: src/backend/modules/runtime/intake/agent_readiness.py line 636 defines score as the percentage of crawlers allowed.
• agentSkills returned pass for dabyte.ai, dablock.ai and vectory.space with details.v2ValidEntries false; isitagentready.com returned v2ValidEntries true with 23 skills. Both indexes are reported as specVersion 0.2.0, and neither file contains a specVersion field.
• Our live https://vectory.space/sitemap.xml declares xmlns="http://www.sitemap.org/schemas/sitemap/0.9" (the registered namespace is sitemaps.org) and the scanner returns "sitemap.xml exists with valid structure" for it. In our own checker, agent_readiness.py returns score 100 with note no_robots_txt_default_allow when robots.txt is absent (lines 661-670), _parse_robots_groups compares only the key "disallow" (line 719), and _check_sitemap validates no namespace (lines 802-806).

The table is published, and the announcement points straight at it

Before writing the first version of this we assumed the check-to-level mapping was undocumented, because it is absent from the results page. That was wrong twice over. The scanner publishes 23 skill files under /.well-known/agent-skills/, and the one named scan-site carries a Level Scale table; fetched on 2026-08-09 it returned 200, text/markdown, 2504 bytes. Cloudflare's 2026-04-17 announcement does not reproduce the table, but it names the skills index and gives its URL, stating that the index carries skill documents for every standard the scanner checks so that agents know not only what to fix but how. The mapping was one hop from the announcement the whole time. As fetched on 2026-08-09 the table reads: level 0 is fewer than two of robots.txt, sitemap and Link headers; level 1 is two of those three; level 2 is level 1 plus both AI bot rules in robots.txt and Content Signals; level 3 is level 2 plus markdown content negotiation; level 4 is level 3 plus one of four integrations (MCP Server Card, A2A Agent Card, agent skills, API catalog); level 5 is level 4 plus two of three: Web Bot Auth, all integrations, auth metadata (OAuth or Auth.md). Read that last line slowly. The parenthesis is a disjunction, and it is where we went wrong. One naming hazard is worth a sentence. The announcement calls the fourth dimension Capabilities; the JSON calls the category discovery; the user interface labels it API / Auth / MCP. The announcement also files Web Bot Auth under Bot Access Control, while the level table treats it as one of the three level-5 items. Three names for one group, and one check that sits in different places depending on which document you are reading.

The ladder is cumulative, so work below your rung does not count yet

vectory.space is the clean demonstration. On 2026-08-09 it passed apiCatalog, mcpServerCard, agentSkills and webMcp, which is the entire level-4 integration group, and it still returned level 2 with a single item in nextLevel: markdownNegotiation. Four integration checks passing bought nothing, because the rung below them was open. The inverse case is www.cloudflare.com, which returned level 3 with markdown negotiation passing and none of the four integrations. Its nextLevel listed all four, though the published table says one is enough. The practical order that follows is the order in the table, not the order of effort. Content Signals in robots.txt is a text edit; markdown negotiation is a server change; the integrations are files you then have to keep in sync with reality. Doing the third before the first leaves the reported level where it was.

nextLevel is a list of your gaps, not a checklist to complete

The field reads like a requirements list, and requirements is the word the JSON uses, but it is the set of checks in the next rung's group that this particular site is failing. On the 3-to-4 step, www.cloudflare.com received four items where one satisfies the rule. On the 4-to-5 step, dabyte.ai, dablock.ai, developers.cloudflare.com, isitagentready.com and settlewithai.com all received the same pair, authMd and a2aAgentCard, because they have the API catalog and lack the same two things. modelcontextprotocol.io received authMd and apiCatalog instead, because it already publishes an A2A agent card and is missing the catalog. The list is computed from your gaps, so two sites on the same rung get different instructions. At level 5 the field is absent entirely. blog.cloudflare.com, sentry.io and turva.dev returned seven top-level keys instead of eight on 2026-08-09, with nextLevel omitted because there is no next rung. Parser code that assumes the key exists will break on exactly the sites at the top of the scale.

The rung we misread, and the check that explains it

Our own handoff note of 2026-08-06 recorded the 4-to-5 gate as authMd plus a2aAgentCard, read off three real nextLevel responses. The first version of this article inherited that reading and then treated blog.cloudflare.com as a contradiction: level 5 while failing both. We proposed DNS-AID as the hidden explanation. That was wrong, and a single site settles it. sentry.io returned level 5 on 2026-08-09 at 14:44 UTC with dnsAid failing and no DNS-AID record at all: a query for _index._agents.sentry.io returns Status 0 with zero answers. It also fails authMd, a2aAgentCard, mcpServerCard and agentSkills, so it does not hold the all-integrations item either. What it passes is oauthProtectedResource and oauthDiscovery, and that satisfies auth metadata through the OAuth branch. The controlled comparison is developers.cloudflare.com, scanned in the same minutes: level 4, and of the six fields that matter it differs from sentry.io on exactly one, oauthProtectedResource. Its nextLevel of authMd and a2aAgentCard, which we had read as a requirement, is simply the two ways it could pick up the item it is missing. Applied to all 23 sites, the published table predicts 23 of 23 returned levels. The same run also confirms the rung at the bottom: sentry.io holds level 5 while failing linkHeaders, so level 1 really is two of three and not all three. The lesson is narrow and cheap. We had a correct table, quoted it correctly, and then reasoned from the fix list instead of from the rule.

A fourth status the documentation does not mention

scan-site/SKILL.md states that each check has a status of pass, fail or unableToCheck. Across 483 check results on 2026-08-09 the API returned four values: fail 202, pass 162, neutral 118, unableToCheck 1. neutral is what the scanner returns when a check does not apply. All five commerce checks came back neutral on the 19 sites it did not classify as commerce, and webBotAuth came back neutral on all 23, with the message "Web Bot Auth directory not found (informational only)" on 22 of them and "Web Bot Auth directory returned HTML instead of JSON (informational only)" on sentry.io. That status is load-bearing for the level-5 arithmetic, and we cannot fully settle how it is counted. Because Web Bot Auth was neutral on every site in the sample and never pass or fail, two readings fit the data identically: either neutral satisfies that item and you need two of three, or the item drops out of the count and you need one of the remaining two. Both predict all 23 levels. The operational consequence is the same either way, which is what matters for planning: while Web Bot Auth stays neutral, moving from 4 to 5 takes exactly one of all four integrations or auth metadata. One branch of that rule is still untested here. Both level-5 sites that lack a complete integration set reached it through OAuth, and turva.dev has everything at once, so no site in this sample reached level 5 on the full-integrations branch alone.

A pass is not a statement of conformance

agentSkills returned pass for dabyte.ai, dablock.ai and vectory.space while the same responses carried details.v2ValidEntries false. We previously wrote that up as our files declaring a stale spec version, and that diagnosis does not survive checking: the scanner reports our index as specVersion 0.2.0, the same value it reports for its own, and neither index file contains a specVersion field at all, so the number is not read from the document. What is observable is entry shape. Our entries use "type": "text/markdown" with an "sha256" field, while the scanner's own index uses "type": "skill-md" with "digest": "sha256:...". Our $schema is https://agentskills.io/schemas/index.json and theirs is https://schemas.agentskills.io/discovery/0.2.0/schema.json. Neither can be read: their host does not resolve (NXDOMAIN on 2026-08-09) and our URL returns 404. So we can see that our file is shaped differently from one the scanner marks v2ValidEntries true, and we cannot consult the spec to learn the required shape. That gap is worth reporting on its own, because it means neither side can validate its index against the schema it declares. Two more from the same day. robotsTxtAiRules returned pass for isitagentready.com, vercel.com and blog.cloudflare.com with "No AI-specific bot rules; wildcard rules apply to all crawlers including AI bots" - a bare wildcard passes, so a pass there is not evidence that anyone enumerated crawlers. And webMcp returned unableToCheck for vercel.com with "Browser session timed out", because that check drives a headless browser and can miss for reasons that have nothing to do with the site. auth.md shows the layering from the other side, with four distinct failure messages across six sites on one day: "auth.md not found" (blog.cloudflare.com), "auth.md returned HTML instead of Markdown" (vectory.space, where a catch-all answers 200 text/html to any path), "auth.md exists but is missing the expected Auth.md heading" (isitagentready.com and sentry.io), and "auth.md exists but does not describe agent registration" (dabyte.ai and dablock.ai). Reachable, correct media type, an H1 containing auth.md, then content that documents registration. Fixing one message reveals the next. Read details and evidence, not status.

The same defect class in our own checker

We hold a scanner to this standard, so here is ours failing it. In src/backend/modules/runtime/intake/agent_readiness.py the robots.txt check returns score 100 with note no_robots_txt_default_allow when the file is missing or non-200 (lines 661-670). The reasoning is correct by the standard, since no file means nothing is disallowed, and the output is still misleading, because a site with no robots.txt also has no Sitemap directive and no Content-Signal line, which is the difference between level 1 and level 2 on the ladder above. _parse_robots_groups compares only the key "disallow" (line 719) and never reads Allow, so the common pair of Disallow: / with Allow: /public/ is scored as a full block. _check_sitemap accepts any 200 whose body starts with <?xml or contains <urlset in the first 200 characters (lines 802-806); it does not validate the namespace and it fetches none of the URLs in the file. Our own published sitemap carries the matching error end to end. marketing/sitemap.xml, served live at https://vectory.space/sitemap.xml on 2026-08-09, declares xmlns="http://www.sitemap.org/schemas/sitemap/0.9", while the registered namespace is sitemaps.org with an s. This is a hand-maintained static file, not generator output: the generator in the codebase emits the correct string at src/backend/modules/greenfield/models.py line 378. The scanner returns "sitemap.xml exists with valid structure" for that file, with details format xml and fromRobotsTxt true. We tested no other tool in this category, so that is one scanner's behaviour on one file, and a question worth putting to any checker you rely on rather than a claim about the class. The general form: a checker that reports a binary status is reporting the presence of a shape, not the correctness of a document. That is true of ours and true of the one this article is about, and it is why the deliverable for a client is the evidence block plus the dated before-and-after scan.json, not the number.

Questions this answers

Machine access

• /api/articles.json — every piece we published, with the measurement each one rests on
• /api/aiv.json — the AI Visibility Index the numbers above come from
• How the index is measured