Rtfm Testing

RTFM Testing

A documentation quality methodology that spawns fresh agents to validate whether docs are actually usable.

The Problem

Documentation written by the person who built the thing is almost always incomplete. They fill in gaps unconsciously. They assume context. They skip "obvious" steps.

RTFM Testing fixes this by spawning a fresh agent with zero context and asking: can you complete this task using only the docs?

When to Use

• Before publishing docs, READMEs, tutorials, or setup guides
• When users report confusion but you can't see why
• After major refactors to validate docs still work
• As part of CI for documentation-heavy projects

How It Works

1. Identify the task — What should someone be able to do after reading the docs?
2. Bundle the docs — Collect all relevant documentation (and nothing else)
3. Spawn a fresh tester — Use the TESTER.md prompt with sessions_spawn
4. Analyze failures — Every confusion point is a doc bug
5. Fix and repeat — Update docs, respawn, retest until clean

Usage

sessions_spawn(
  task: "Complete the following task using ONLY the provided documentation. [TASK DESCRIPTION]\n\n---\n\n[PASTE DOCS HERE]",
  agentId: "default",
  label: "rtfm-test"
)

Or use the full TESTER.md prompt for more structured output.

Metrics

• Cold Start Score — Number of spawn cycles until task completion (lower = better docs)
• Gap Count — Number of [GAP] reports per run
• Gap Categories — Missing steps, unclear language, wrong assumptions, missing prerequisites

Key Principles

1. No hints — Don't help the tester. Let it fail.
2. Literal reading — Tester must not infer or guess
3. Docs only — No external knowledge, no "common sense"
4. Failures are signal — Every stumble is actionable feedback

Files

• SKILL.md — This file
• TESTER.md — System prompt for the fresh agent
• GAPS.md — Output format specification