Search Agentized Websites

Search Agentized Websites

Treat Agentize as a publisher-controlled HTTP interface. The target agent does not need the Agentize package or its source code.

Detect Agentize

Given a website origin, request GET <origin>/agents with Accept: application/json.

Call the site Agentized only when all of these hold:

• The response is successful JSON, not an HTML page or redirect destination masquerading as discovery.
• version is a supported protocol version; this skill supports 0.1 and 0.2.
• A 0.2 manifest contains access.mode set to public or protected.
• site contains a non-empty name, origin, and description.
• index, search, and resources are URL strings; the resources value contains {id}.
• Resolved endpoints use HTTP(S) and remain on the declared site origin. Do not follow a manifest to another origin unless the user explicitly authorizes it.

A 404, HTML response, malformed manifest, unsupported version, or network failure means “Agentize not detected,” with the reason. A 401 means the interface requires host credentials. Report that requirement when credentials are unavailable; do not probe alternate paths or attempt a bypass.

Use credentials supplied by the user or execution environment. Send them on discovery and every later Agentize request. Never forward credentials across origins or print their values.

For terminal work, use the dependency-free helper:

python3 scripts/agentize_client.py detect https://example.com

The helper reads sensitive headers from environment variables. Put global options before the command:

export COMPANY_AGENT_TOKEN='Bearer ...'
python3 scripts/agentize_client.py \
  --header-env Authorization=COMPANY_AGENT_TOKEN \
  detect https://intranet.example

Search with minimum requests

When the user asks to search a site, make discovery and one semantic search the normal path. Do not fetch the site index first.

1. Reuse a validated manifest already obtained for the same origin during the task. Otherwise discover it once.
2. Send one POST to the discovered search endpoint with a focused natural-language query and limit of 3–5.
3. Preserve exact SKUs, error codes, API symbols, plan names, and quoted identifiers in the query because Agentize deliberately boosts exact identifier matches.
4. Judge results by title, description, and matched passages. Scores are meaningful only within that response; never compare scores across websites or searches.
5. Retrieve the normalized Markdown only for the result or small number of results needed to answer. Use the opaque resourceId returned by search, URI-encode it as one path segment, and request Accept: text/markdown.

Use /agents/index only when collection discovery would improve a query, such as when the user names an unfamiliar section or the first search is too broad. It lists collections and important standalone resources; it is not a sitemap. Do not paginate it unless the standalone resource listing is actually needed.

Use filters only when their values are known from the manifest index, a prior response, or the user. Filter arrays are OR within one field and AND across fields. Never invent collection, type, tag, or language values.

If the first search has no useful result, make at most one deliberate retry: remove an uncertain filter or rephrase the same intent with likely publisher terminology. For genuinely independent intents, send separate searches and run them in parallel when the available HTTP tool permits. Do not combine unrelated questions into one oversized query.

The helper performs discovery and search in one invocation:

python3 scripts/agentize_client.py search https://example.com \
  "Can employees sign in with the company identity provider?" --limit 5

For an authenticated interface, pass the same environment-backed header option before search or get. The helper applies it to discovery and the requested operation.

Optional filters repeat as needed:

python3 scripts/agentize_client.py search https://example.com \
  "How do I rotate an API key?" \
  --collection docs --type documentation --language en

Retrieve and answer

Prefer the Agentize resource representation over crawling the result's canonical page. It is the publisher-declared, normalized content intended for agents. Fetch JSON only when metadata, tags, relationships, or canonical route fields matter; otherwise Markdown is smaller and ready to reason over.

python3 scripts/agentize_client.py get https://example.com 'docs:api-keys'
python3 scripts/agentize_client.py get https://example.com 'docs:api-keys' --json

Preserve source attribution using the result's canonical url, not the /agents/resources/* transport URL. Distinguish what the source states from inference. If multiple Agentized sites are searched, attribute each answer to its publisher and do not merge their relevance scores.

Protocol details and failures

Read references/protocol-v0.2.md for current public and protected interfaces. Read references/protocol-v0.1.md only when the manifest reports 0.1.

Agentize never accepts a URL to fetch. Do not send page URLs in place of a query or resource ID. Treat resourceId values as opaque, honor ETag/If-None-Match when revisiting resources, and stop rather than bypassing authentication or access controls.