# Is Your Site Agent-Ready? -- Full Documentation

A scanner that checks whether websites implement the standards and protocols that make them accessible to AI agents. Scans 18 checks across 5 categories and assigns a readiness level from 0 (Not Ready) to 5 (Agent-Native).

## API

### Scan Endpoint

```
POST https://isitagentready.com/api/scan
Content-Type: application/json

{"url": "https://example.com"}
```

#### Parameters

| Parameter | Type   | Required | Description |
|-----------|--------|----------|-------------|
| url       | string | Yes      | The URL to scan. Must be a valid HTTP or HTTPS URL. |
| format    | string | No       | Response format: `"json"` (default) or `"agent"`. |

#### JSON Response Shape

```json
{
  "url": "https://example.com",
  "scanned_at": "2025-01-01T00:00:00.000Z",
  "level": 3,
  "level_name": "Agent-Readable",
  "checks": {
    "discoverability": {
      "robotsTxt": { "status": "pass", "message": "..." },
      "sitemap": { "status": "pass", "message": "..." },
      "linkHeaders": { "status": "fail", "message": "..." }
    },
    "contentAccessibility": {
      "markdownNegotiation": { "status": "pass", "message": "..." }
    },
    "botAccessControl": {
      "robotsTxtAiRules": { "status": "pass", "message": "..." },
      "contentSignals": { "status": "pass", "message": "..." },
      "webBotAuth": { "status": "neutral", "message": "..." }
    },
    "discovery": {
      "mcpServerCard": { "status": "pass", "message": "..." },
      "a2aAgentCard": { "status": "fail", "message": "..." },
      "agentSkills": { "status": "pass", "message": "..." },
      "webMcp": { "status": "neutral", "message": "..." },
      "apiCatalog": { "status": "fail", "message": "..." },
      "oauthDiscovery": { "status": "fail", "message": "..." },
      "oauthProtectedResource": { "status": "fail", "message": "..." }
    },
    "commerce": {
      "x402": { "status": "neutral", "message": "..." },
      "ucp": { "status": "neutral", "message": "..." },
      "acp": { "status": "neutral", "message": "..." },
      "ap2": { "status": "neutral", "message": "..." }
    }
  },
  "next_level": {
    "target": 4,
    "name": "Agent-Integrated",
    "requirements": [...]
  },
  "is_commerce": false,
  "commerce_signals": []
}
```

Each check returns a `status` of `"pass"`, `"fail"`, `"neutral"`, or `"unableToCheck"`, plus a human-readable `message` and optional `details` object.

#### Agent Response Format

When `format: "agent"` is used, the response is a `text/markdown` document listing all failing checks with fix instructions. This is designed for LLM consumption.

### MCP Endpoint

```
POST https://isitagentready.com/mcp
```

Streamable HTTP MCP server. Supports the `scan_site` tool which accepts a `url` parameter and returns the same scan results.

## The 18 Checks

### Discoverability

#### 1. robots.txt (`robotsTxt`)
- **What**: Published /robots.txt with clear crawl rules
- **Pass criteria**: Returns 200 with text/plain content containing at least one User-agent directive
- **Fix**: Create /robots.txt at the site root with explicit User-agent directives and allow/disallow rules for key paths. Ensure it is plain text and returns 200.
- **Spec**: https://www.rfc-editor.org/rfc/rfc9309

#### 2. Sitemap (`sitemap`)
- **What**: Published /sitemap.xml with canonical URLs
- **Pass criteria**: /sitemap.xml exists and returns valid XML, or a Sitemap directive is found in robots.txt
- **Fix**: Generate /sitemap.xml listing canonical URLs, keep it updated on publish, and reference it from /robots.txt.
- **Spec**: https://www.sitemaps.org/protocol.html

#### 3. Link Headers (`linkHeaders`)
- **What**: Link response headers for agent discovery (RFC 8288)
- **Pass criteria**: Homepage includes Link response headers with agent-useful relation types (service-desc, describedby, api-catalog, etc.)
- **Fix**: Add Link response headers to your homepage that point agents to useful resources such as API catalogs, service descriptions, or documentation.
- **Spec**: https://www.rfc-editor.org/rfc/rfc8288, https://www.rfc-editor.org/rfc/rfc9727#section-3

### Content Accessibility

#### 4. Markdown Negotiation (`markdownNegotiation`)
- **What**: Content negotiation for markdown via Accept header
- **Pass criteria**: Requesting the homepage with `Accept: text/markdown` returns a response with `Content-Type: text/markdown`
- **Fix**: Implement content negotiation so requests with Accept: text/markdown return a markdown representation while HTML remains the default for browsers.
- **Spec**: https://developers.cloudflare.com/fundamentals/reference/markdown-for-agents/

### Bot Access Control

#### 5. AI Bot Rules (`robotsTxtAiRules`)
- **What**: User-agent rules for AI crawlers in robots.txt
- **Pass criteria**: robots.txt contains User-agent entries for known AI bots (GPTBot, OAI-SearchBot, Claude-Web, Google-Extended, etc.) or wildcard rules that apply to all crawlers
- **Fix**: Add explicit User-agent entries for AI crawlers (GPTBot, OAI-SearchBot, Claude-Web, Google-Extended) with allow/disallow rules that match your policy.
- **Spec**: https://www.rfc-editor.org/rfc/rfc9309, https://developers.cloudflare.com/ai-crawl-control/

#### 6. Content Signals (`contentSignals`)
- **What**: AI content usage preferences declared in robots.txt
- **Pass criteria**: robots.txt contains Content-Signal directives with ai-train, search, or ai-input preferences
- **Fix**: Add Content-Signal directives to your robots.txt declaring preferences for ai-train, search, and ai-input. For example: Content-Signal: ai-train=no, search=yes, ai-input=no
- **Spec**: https://contentsignals.org/, https://datatracker.ietf.org/doc/draft-romm-aipref-contentsignals/

#### 7. Web Bot Auth (`webBotAuth`)
- **What**: HTTP message signature verification for bots (informational)
- **Pass criteria**: /.well-known/http-message-signatures-directory exists with valid JWKS
- **Note**: This check is informational only. Most sites receive bot traffic but don't send it, so JWKS-based identity proof is not broadly applicable. Failures are shown as neutral and do not affect the score. Passes are shown as green to acknowledge implementation.
- **Spec**: https://datatracker.ietf.org/wg/webbotauth/about/, https://developers.cloudflare.com/bots/reference/bot-verification/web-bot-auth/

### Discovery

#### 8. MCP Server Card (`mcpServerCard`)
- **What**: MCP Server Card for agent discovery (SEP-1649)
- **Pass criteria**: A valid MCP Server Card exists at /.well-known/mcp/server-card.json, /.well-known/mcp/server-cards.json, or /.well-known/mcp.json with serverInfo.name or a flat name field
- **Fix**: Serve an MCP Server Card at /.well-known/mcp/server-card.json with serverInfo (name, version), transport endpoint, and capabilities.
- **Spec**: https://github.com/modelcontextprotocol/modelcontextprotocol/pull/2127

#### 9. A2A Agent Card (`a2aAgentCard`)
- **What**: A2A Agent Card for agent-to-agent discovery (opt-in)
- **Pass criteria**: /.well-known/agent-card.json exists with name, version, and supportedInterfaces
- **Fix**: Serve an A2A Agent Card at /.well-known/agent-card.json describing your agent's name, version, capabilities, supported interfaces, and skills.
- **Spec**: https://a2a-protocol.org/latest/specification/

#### 10. Agent Skills (`agentSkills`)
- **What**: Agent skills discovery document
- **Pass criteria**: /.well-known/agent-skills/index.json exists with a valid skills array (v0.2.0 path; legacy /.well-known/skills/index.json also accepted)
- **Fix**: Publish a skills discovery index at /.well-known/agent-skills/index.json with $schema, and a skills array (name, type, description, url, digest per entry).
- **Spec**: https://github.com/cloudflare/agent-skills-discovery-rfc

#### 11. WebMCP (`webMcp`)
- **What**: WebMCP tools exposed via browser JavaScript API
- **Pass criteria**: Page exposes MCP tools via the WebMCP JavaScript API
- **Fix**: Implement the WebMCP API in your client-side JavaScript to expose tools that AI agents can discover and invoke.
- **Spec**: https://webmachinelearning.github.io/webmcp/

#### 12. API Catalog (`apiCatalog`)
- **What**: API catalog for automated API discovery (RFC 9727)
- **Pass criteria**: /.well-known/api-catalog returns valid linkset+json with a "linkset" array containing API entries
- **Fix**: Create /.well-known/api-catalog returning application/linkset+json with a "linkset" array. Each entry should include an "anchor" URL for the API and link relations for service-desc, service-doc, and status.
- **Spec**: https://www.rfc-editor.org/rfc/rfc9727, https://www.rfc-editor.org/rfc/rfc9264

#### 13. OAuth Discovery (`oauthDiscovery`)
- **What**: OAuth/OIDC discovery metadata for agent authentication
- **Pass criteria**: /.well-known/openid-configuration or /.well-known/oauth-authorization-server exists with valid OAuth metadata
- **Fix**: Publish OAuth/OIDC discovery metadata with issuer, authorization_endpoint, token_endpoint, and grant_types_supported.
- **Spec**: http://openid.net/specs/openid-connect-discovery-1_0.html, https://www.rfc-editor.org/rfc/rfc8414

#### 14. OAuth Protected Resource (`oauthProtectedResource`)
- **What**: OAuth Protected Resource Metadata (RFC 9728)
- **Pass criteria**: /.well-known/oauth-protected-resource exists with resource identifier and authorization_servers
- **Fix**: Publish /.well-known/oauth-protected-resource with resource, authorization_servers, and scopes_supported.
- **Spec**: https://www.rfc-editor.org/rfc/rfc9728

### Commerce

Commerce checks are only scored for sites with detected e-commerce signals. Non-commerce sites receive neutral results.

#### 15. x402 (`x402`)
- **What**: x402 payment protocol for agent-native HTTP payments
- **Pass criteria**: API routes return HTTP 402 with valid x402 payment headers
- **Fix**: Add x402 payment middleware to your API routes to enable agent-native HTTP payments.
- **Spec**: https://x402.org, https://docs.x402.org

#### 16. UCP (`ucp`)
- **What**: Universal Commerce Protocol profile
- **Pass criteria**: /.well-known/ucp exists with protocol_version and services
- **Fix**: Serve /.well-known/ucp with protocol version, services, capabilities, and endpoints.
- **Spec**: https://ucp.dev/specification/overview/

#### 17. ACP (`acp`)
- **What**: Agentic Commerce Protocol discovery document
- **Pass criteria**: /.well-known/acp.json exists with protocol.name, protocol.version, api_base_url, transports, and capabilities.services
- **Fix**: Serve /.well-known/acp.json with protocol metadata so agents can discover your ACP implementation.
- **Spec**: https://agenticcommerce.dev

#### 18. AP2 (`ap2`)
- **What**: Agent Payments Protocol support via A2A Agent Card extensions (opt-in)
- **Pass criteria**: A2A Agent Card includes an AP2 extension with role information
- **Fix**: Declare AP2 support in your A2A Agent Card extensions with your role (merchant, shopper, etc.).
- **Spec**: https://ap2-protocol.org/

## Level System

Sites are scored on a 0-5 scale. Each level builds on the previous:

### Level 0 -- Not Ready
Does not meet Level 1 criteria.

### Level 1 -- Basic Web Presence
Pass 2 of 3: robots.txt, sitemap, link headers.

### Level 2 -- Bot-Aware
Level 1 + both: AI bot rules and Content Signals in robots.txt.

### Level 3 -- Agent-Readable
Level 2 + markdown content negotiation.

### Level 4 -- Agent-Integrated
Level 3 + 1 of 4: MCP Server Card, A2A Agent Card, Agent Skills, API Catalog.

### Level 5 -- Agent-Native
Level 4 + 2 of 3: Web Bot Auth, all four integration checks, auth metadata (OAuth discovery or OAuth Protected Resource).

## Links

- Site: https://isitagentready.com
- MCP Endpoint: https://isitagentready.com/mcp
- API Catalog: https://isitagentready.com/.well-known/api-catalog
- Agent Skills: https://isitagentready.com/.well-known/agent-skills/index.json
- Cloudflare Agents docs: https://developers.cloudflare.com/agents/
- MCP spec: https://modelcontextprotocol.io/
- Content Signals spec: https://contentsignals.org/
- RFC 9309 (robots.txt): https://www.rfc-editor.org/rfc/rfc9309
- RFC 9727 (API Catalog): https://www.rfc-editor.org/rfc/rfc9727
- RFC 9728 (OAuth Protected Resource): https://www.rfc-editor.org/rfc/rfc9728
- Agent Skills Discovery: https://github.com/cloudflare/agent-skills-discovery-rfc