zscole RTFM Testing
A documentation quality methodology that spawns fresh agents to validate whether docs are actually usable.
"If an amnesiac can't follow your docs, your docs suck."
The Problem
Documentation written by the person who built the thing is almost always incomplete. They fill in gaps unconsciously. They assume context. They skip "obvious" steps.
RTFM Testing fixes this by spawning a fresh agent with zero context and asking: can you complete this task using only the docs?
When to Use
- Before publishing docs, READMEs, tutorials, or setup guides
- When users report confusion but you can't see why
- After major refactors to validate docs still work
- As part of CI for documentation-heavy projects
How It Works
- Identify the task — What should someone be able to do after reading the docs?
- Bundle the docs — Collect all relevant documentation (and nothing else)
- Spawn a fresh tester — Use the TESTER.md prompt with
sessions_spawn - Analyze failures — Every confusion point is a doc bug
- Fix and repeat — Update docs, respawn, retest until clean
Usage
sessions_spawn(
task: "Complete the following task using ONLY the provided documentation. [TASK DESCRIPTION]\n\n---\n\n[PASTE DOCS HERE]",
agentId: "default",
label: "rtfm-test"
)
Or use the full TESTER.md prompt for more structured output.
Metrics
- Cold Start Score — Number of spawn cycles until task completion (lower = better docs)
- Gap Count — Number of
[GAP]reports per run - Gap Categories — Missing steps, unclear language, wrong assumptions, missing prerequisites
Key Principles
- No hints — Don't help the tester. Let it fail.
- Literal reading — Tester must not infer or guess
- Docs only — No external knowledge, no "common sense"
- Failures are signal — Every stumble is actionable feedback
Files
SKILL.md— This fileTESTER.md— System prompt for the fresh agentGAPS.md— Output format specification