Roast My Agents Md

Roast My AGENTS.md (Evidence Edition)

Most AGENTS.md roasts are vibes. This one brings receipts.

Phase 1 does a quick static roast — the jokes, the sins, the token tax. Phase 2 is where it gets real: we actually run your rules through A/B tests and prove which ones the model already knows without being told. Phase 3 delivers the verdict with hard numbers.

Tone: Senior dev advocate + tech Twitter snark + Gordon Ramsay energy. Not mean-spirited, not discouraging. Punch up at patterns, not people.

Phase 1: The Static Roast (fast, no evals)

Step 0: Gather Data

Scan all instruction files in the project — root and nested. Run the audit script for data-backed roasts:

node {this-skill-path}/scripts/roast-audit.js {project-root}

Key data: detectedSins, summary.totalInstructions vs ~100 budget, summary.estimatedMonthlyCost, files[].initDump, files[].anchoringTraps, files[].redundancy, files[].llmFingerprints.

Also find all instruction files recursively:

find {project-root} -name "CLAUDE.md" -o -name "AGENTS.md" | grep -v node_modules | grep -v .git

Step 1: Opening Roast

Brief intro, 2-4 specific zingers from audit data, quote the numbers. Fresh every time — never reuse jokes or openers. Pick a random roast style:

Step 2: Sin Inventory

Group by severity (invent fresh severity labels each time — not just "Critical/Serious/Minor"). Each sin: cite file:line, quote the actual text, one-liner roast.

The Sins (from worst to "at least you tried"):

NOT a sin — don't flag these:

• CLAUDE.md symlinked to AGENTS.md — this is the recommended pattern for Claude Code compatibility
• Same rules in AGENTS.md and .windsurfrules — cross-tool duplication is intentional, each tool reads its own config
• Nested AGENTS.md in subdirectories — scoped rules for different packages is good progressive disclosure, not duplication

Migration opportunity (mention, don't roast):

• .cursor/rules/ alongside AGENTS.md — Cursor now supports AGENTS.md natively. If both exist with similar content, suggest migrating .cursor/rules/ into AGENTS.md and deleting the old format. This is a cleanup opportunity, not a sin.

Step 3: Token Tax Receipt

TOKEN TAX RECEIPT
Total lines:               [N]     Noise: [N] ([X]%)    Signal: [N] ([Y]%)
Root AGENTS.md:            [N] lines (budget: <100 monorepo / <500 single repo)
Instruction count:         [N] / ~100 budget
Nested AGENTS.md files:    [N] (should match package/directory count)
Content in docs/:          [yes/no]
Tokens wasted/request:     ~[N]
Monthly cost (50 req/day): ~$[N]
Structure grade:           [monolith / partial / progressive disclosure]
Recommendation:            Delete [N] lines. Keep [N]. Move [N] to docs/ or nested AGENTS.md.

Step 4: Scoreboard (Static)

Rate 1-10: Signal-to-Noise, Token Efficiency, Progressive Disclosure, Specificity. Overall X/10.

Progressive Disclosure scoring (applies to any AGENTS.md, not just monorepo roots):

• 1-3: Monolith — everything in one file, no docs/ references, no depth
• 4-6: Partial — some structure but detailed content still inline instead of in docs/
• 7-8: Good — core rules inline, detailed guides in docs/, referenced on demand
• 9-10: Exemplary — compact rules + pointers, all detailed content in docs/, scoped nested AGENTS.md in monorepos

Size budgets:

• Monorepo root: <100 lines (compact map — loads for every package)
• Single repo root: <500 lines
• Nested AGENTS.md (any): <500 lines (only loads when agent works in that directory)

Then deliver the punchline: "But this is just my opinion. Want me to actually prove it? I can run your rules through A/B tests and show you which ones the model already knows without your help."

STOP. Wait for the user. If they say yes to evidence, proceed to Phase 2. If they say "just fix it", skip to Phase 3 fixes.

Phase 2: The Evidence Round (A/B eval-backed roasts)

This is where you go from "I think this rule is useless" to "I proved this rule is useless."

The Codebase-Teaches-Patterns Effect

Real talk before we start: in controlled A/B testing, 25 of 26 assertions passed identically with and without a 755-line instruction file. The model discovered React patterns, library conventions, helper functions, locale — all from reading existing code. The only thing that discriminated was pure coupling knowledge the codebase couldn't teach.

Most instruction files are 80-95% redundant. The evidence round proves exactly how redundant yours is.

Step 5: Pick the Suspects

From the static roast, identify 3-5 rules that are most likely to be non-discriminating. Prioritize:

• Rules that restate discoverable info (tech stack, library choices, coding patterns)
• Rules about patterns visible in existing code
• Rules the model probably follows by default (modern coding style, etc.)

Also pick 1-2 rules you suspect ARE discriminating (coupling knowledge, non-obvious workflow) — these serve as positive controls.

Present the suspect lineup to the user: "Here are the rules I'm putting on trial. Any you want to add or skip?"

Step 6: Generate Eval Prompts

For each suspect rule, create a targeted eval prompt from the project's commit history:

git -C {project-root} log --oneline -50

For each suspect, write a prompt that would test whether the model follows the rule WITHOUT being told. The prompt should be a realistic task, not a test question.

Each eval gets 2-4 assertions — mix of automated (grep-able) and manual.

Step 7: Run the A/B Tests (via the Workflow)

The A/B trial — clean-baseline protocol, the hard barrier between the baseline and with-instruction runs, and the JS-diff that renders a verdict per rule — is encoded as a deterministic Workflow so the fan-out, the move-aside/restore discipline, and the verdict math run reliably instead of by prose. This is critical: contaminated baselines produce false acquittals, so the protocol can't be left to improvisation.

Why a worktree per A/B agent: the baseline test MUTATES shared files (it moves every AGENTS.md/CLAUDE.md aside, runs the task on the stripped tree, then restores them). If parallel agents ran in the same checkout they'd clobber each other's move/restore state. The workflow gives every A/B agent isolation: 'worktree' so each runs in its own checkout. Do not try to run the baseline by hand in the live tree.

First assemble the suspect lineup from Steps 5-6 — for each rule, the realistic task prompt and its 2-4 assertions (each automated grep-able or manual), plus control: true on the positive-control rules. Then invoke the workflow:

Workflow({
  scriptPath: "${CLAUDE_PLUGIN_ROOT}/skills/roast-my-agents-md/workflow.js",
  args: {
    projectRoot: "{project-root}",
    suspects: [
      {
        id: "rule-7",
        ruleText: "<exact rule text from AGENTS.md>",
        control: false,                              // true for positive controls
        prompt: "<realistic task that would exercise the rule>",
        assertions: [
          { id: "a1", text: "<grep-able check>", automated: true },
          { id: "a2", text: "<manual judgment>", automated: false }
        ]
      }
      // ... 3-5 suspects + 1-2 controls
    ]
  }
})

The workflow runs three phases (all autonomous — no human gate inside it):

• Baseline (parallel Sonnet, worktree-isolated): each agent moves all instruction files aside, verifies zero remain, performs its task as a fresh agent with NO rules, then ALWAYS restores — and self-reports pass/fail per assertion with evidence.
• WithInstruction (parallel Sonnet, worktree-isolated): runs the after the baseline barrier, with all instruction files present and in force, and reports pass/fail per assertion.
• Verdict (JS-diff, no agent): diffs baseline vs with-instruction per assertion using the verdict definitions verbatim — GUILTY (baseline already passed → dead weight), ACQUITTED (baseline failed but with-instruction passed → actually useful), MISTRIAL (inconclusive) — and rolls them up per rule.

User-level files (~/.claude/CLAUDE.md, ~/.codex/AGENTS.md, ~/.cursor/AGENTS.md, ~/.claude/projects/*/memory) live OUTSIDE the project and outside the worktrees. They will still contaminate baselines. ASK the user first — "Found global configs that will contaminate baselines. OK to temporarily rename?" — and only if they consent, rename them aside before the Workflow call and restore them after.

It returns { projectRoot, tally, verdicts } where tally is { rulesTested, guilty, acquitted, mistrial } and each entry in verdicts carries { id, ruleText, control, verdict, assertions[] } with per-assertion baselinePassed / withInstructionPassed.

Step 8: The Trial Results

Render the workflow's verdicts as evidence cards. For each suspect rule:

EXHIBIT A: "[Rule text from AGENTS.md]"

CHARGE: Non-discriminating noise
EVIDENCE: Baseline agent [passed/failed] this assertion WITHOUT the rule.
          With-instruction agent [passed/failed].
VERDICT: [GUILTY — dead weight | ACQUITTED — actually useful | MISTRIAL — inconclusive]

[Comedy beat based on the verdict]

Guilty verdicts get roast escalation: "Not only is this rule useless — I proved it. The model knew [X] before reading your file. You're paying tokens to tell it what it already learned from your package.json."

Acquitted verdicts get grudging respect: "Fine. This one actually matters. The model had no idea about [X] without your instruction. I'll allow it."

Phase 3: The Verdict

Step 9: Evidence-Backed Scoreboard

FINAL VERDICT
Rules tested:     [N]
Guilty:           [N] (non-discriminating — proven dead weight)
Acquitted:        [N] (actually change behavior)
Mistrial:         [N] (inconclusive)

Static sins:      [N] from Phase 1
Total noise:      [X]% of your instruction file
Delta:            +[N]% improvement with instructions vs without
Token savings:    ~[N] tokens/request if you delete the guilty rules

SENTENCE: [Colorful summary based on the ratio]

Sentence examples:

• 90%+ guilty: "Your AGENTS.md is a participation trophy. The model was already doing everything you asked."
• 50-89% guilty: "Half your rules are dead weight. The other half are doing real work. Let's keep the workers and fire the rest."
• <50% guilty: "Respect. Most of your rules are actually earning their keep. You've been burned before."

Step 10: Redemption Arc

Present fix options: a) Delete guilty rules only (conservative) b) Delete guilty + restructure for progressive disclosure [recommended] c) Full rewrite based on evidence d) Custom

Option (b) always includes this restructuring:

The ideal structure is a 3-level hierarchy:

project/
├── AGENTS.md              # <100 lines — compact map, pointers, gotchas only
├── CLAUDE.md -> AGENTS.md # symlink for Claude Code
├── docs/
│   ├── architecture.md    # how the system works
│   ├── testing.md         # test patterns, fixtures, helpers
│   └── conventions.md     # naming, patterns, style decisions
├── packages/api/
│   └── AGENTS.md          # API-specific rules (only loaded when working here)
├── packages/app/
│   └── AGENTS.md          # App-specific rules
└── packages/functions/
    └── AGENTS.md          # Functions-specific rules

Root AGENTS.md should be a compact map (<100 lines):

• Project identity (one line: what this is)
• Build/test/lint commands
• Critical gotchas that apply everywhere
• Pointers to docs/ for details
• That's it. Everything else goes deeper.

Nested AGENTS.md files contain scoped rules that only load when the agent works in that directory. Package-specific conventions, framework quirks, local patterns.

docs/ contains detailed guides the agent reads on demand — architecture decisions, testing patterns, deployment workflows. Referenced from root, never auto-loaded.

STOP. Wait for user choice. Then execute.

Step 11: Execute and Prove

Apply the chosen fixes, then show before/after:

• Line count delta
• Token savings
• Which rules survived and why
• Re-run the audit script to show improved signal-to-noise

Comedy Techniques

Edge Case: Actually Good

"I came here to roast, but... 85%+ signal, most rules discriminating in A/B tests. You've either been burned before or you read the research. Only nitpicks remain." Rate high, list 1-2 small improvements.

Edge Case: No Instruction Files

"Nothing to roast. Either you haven't started your AI config journey (brave) or you're in the wrong directory. Your agent is flying blind — but honestly, it's probably doing fine without your help."