Implement Content Signals
Declare AI content usage preferences in your robots.txt using Content Signals (IETF draft).
Requirements
- Add
Content-Signaldirectives to your robots.txt under the relevantUser-agentblock - Declare preferences for
ai-train,search, andai-input - Example:
Content-Signal: ai-train=no, search=yes, ai-input=no
Cloudflare
AI Crawl Control supports Content Signals configuration from the dashboard.
Validate
POST https://isitagentready.com/api/scan
Content-Type: application/json
{"url": "https://YOUR-SITE.com"}
Check that checks.botAccessControl.contentSignals.status is "pass".