Cloudflare’s Is It Agent Ready? A Practical Guide to the New Agent Readiness Score

The web already learned how to talk to browsers and to search crawlers. The next shift is teaching it to cooperate cleanly with AI agents: automated clients that discover capabilities, negotiate formats, respect crawl policy, and sometimes even pay or authenticate on behalf of users.

In April 2026, Cloudflare shipped isitagentready.com—a free scanner that grades your site on how well it supports those emerging patterns. It is part of a broader push Cloudflare calls preparing for the agentic Internet, alongside Radar datasets and tighter AI Crawl Control tooling.

If you build or maintain marketing sites, documentation, or APIs, this score is a useful compass—not because you need a perfect grade tomorrow, but because it surfaces concrete gaps between “works in Chrome” and “works for an autonomous client.”

What the scanner actually checks

When you enter a URL at isitagentready.com, Cloudflare probes your site for support across several buckets that mirror how agents really behave: discovery first, content shape second, policy and identity third, protocols fourth, and (optionally) commerce-related signals.

On the tool itself, checks are grouped into areas such as Discoverability, Content Accessibility, Bot Access Control, API / Auth / MCP, and Commerce, with toggles so you can scope a scan toward a content site versus an API-oriented surface.

Common highlights include:

• Discoverability — A valid robots.txt, XML sitemaps, and machine-friendly hints such as Link response headers that point agents at catalogs or related resources without parsing HTML first.
• Content accessibility — Whether your origin can serve Markdown via content negotiation (Accept: text/markdown), which Cloudflare has championed as “Markdown for agents”: leaner payloads and fewer tokens than scraping arbitrary HTML. The scanner also lets you opt into checks like llms.txt if you want that signal in scope.
• Bot access control — Explicit AI-oriented crawl rules, adoption of Content Signals (declaring what AI may do with your content), and emerging Web Bot Auth mechanics so good bots can identify themselves verifiably.
• Protocol discovery — Practical glue for agents: MCP server cards, Agent Skills indexes, API catalogs and OAuth discovery endpoints—things that answer “what can I call here?” without hunting through pages of marketing copy.

Cloudflare’s accompanying announcement explains that commerce-oriented protocols (such as x402, UCP, and agentic commerce experiments) may appear in a scan but do not currently affect the headline score—a helpful distinction if you are grading a brochure site versus a transactional API.

Why Cloudflare says most sites are not “agent-ready” yet

To ground the tool in real-world baselines, Cloudflare Radar sampled on the order of 200,000 highly visited domains (with noisy categories filtered out) and published adoption charts—now updated weekly—in Radar AI Insights.

A few results from that analysis are humbling and actionable at the same time:

• robots.txt is common, but often written for legacy crawlers — Roughly four in five sites publish a robots.txt, yet many files still read like classic search-engine era directives rather than policies tuned for AI agents and modern bot families.
• Content Signals are rare — Only about 4% of sites had declared AI usage preferences via Content Signals in robots.txt when the dataset was first discussed—understandable while the standard is young, but a clear place to differentiate.
• Markdown negotiation is still unusual — On the order of 4% of sites passed Markdown negotiation checks—unsurprising, since it asks your stack to serve an alternate representation intentionally, not accidentally.
• Cutting-edge discovery files are extremely scarce — Combined adoption of newer discovery artifacts like MCP Server Cards and API Catalogs appeared on fewer than fifteen sites in the entire sampled set when the blog post shipped. That is not a reason to panic; it is an argument for early, intentional experiments if agent traffic matters to your roadmap.

None of this means your site is “bad.” It means the ecosystem is early, and small, deliberate upgrades can put you ahead of the median experience.

How this connects to products you may already use

Cloudflare tied the launch to practical surfaces you can adopt incrementally:

• AI Crawl Control now links out from the Directives tab (formerly labeled around robots.txt) so you can jump straight from crawl configuration to an Agent Readiness check—useful when you are iterating access rules and want immediate feedback.
• Radar exposes weekly adoption metrics so you can see whether a convention you care about is crossing from niche to mainstream.
• Cloudflare positions isitagentready.com alongside documentation on Cloudflare Agents, reinforcing that readiness is not only about passive crawling—it is about interoperable agents operating safely at scale.

Interestingly, Cloudflare notes that isitagentready.com itself is built to practice what it preaches—for example by exposing agent-centric discovery endpoints so compatible tools can reason about fixes programmatically.

Where we recommend teams start

You do not need to implement every protocol on day one. Cloudflare’s own guidance on the scanner emphasizes easy wins:

1. Publish a truthful, complete robots.txt — Include sitemap references and explicit guidance for AI-oriented crawlers where policies matter to your brand.
2. Ensure discovery paths are real — Broken sitemaps and contradictory directives undermine both SEO and agent traversal.
3. Expose helpful discovery metadata — Headers and well-known URLs that route agents to catalogs or policies beat burying critical links only inside dense HTML.

From there, prioritize based on risk and reward:

• Marketing and editorial sites — Evaluate Markdown negotiation or curated agent-facing summaries where token efficiency genuinely helps support and sales workflows grounded in your content.
• APIs and developer products — Invest in machine-readable catalogs and OAuth discovery if agents are—or will be—first-class API consumers.
• Brand-sensitive publishing — Layer in Content Signals and thoughtful crawl rules so your preferences are explicit rather than implied.

Closing thought

Search optimization asked sites to be legible to crawlers and humans at the same time. Agent readiness adds another lens: your site as a negotiation surface for automated clients that need structure, policy, and capability hints upfront.

Is it agent ready? is a blunt question. The score is not a trophy—it is a checklist for a web where agents are not a hypothetical traffic segment, but a normal part of how people research, build, and buy.

Further reading

• Cloudflare announcement: Introducing the Agent Readiness score (April 17, 2026)
• Product changelog: Tools to prepare your site for the agentic Internet
• Scanner: isitagentready.com