# Cold Tester

> Test a document, runbook, tutorial, or onboarding guide by having a SEPARATE AI with zero prior context try to follow it — like a real new hire who only has the doc. Your main agent stays the approver and rules each step pass/reject, surfacing exactly where the doc is unclear. Use when you want to verify a doc is actually followable, when you say "/cold-tester", "test this doc", "could a newcomer follow this", "try it with someone who's never seen it".

- Skill: `gyujeongion/cold-tester` (Agent Skill, multi-file: 2 files)
- Install (CLI): `npx skillmds@latest add gyujeongion/cold-tester`
- Raw SKILL.md: https://api.skillmd.com/api/skills/gyujeongion/cold-tester/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: gyujeongion (https://skillmd.com/u/gyujeongion)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/gyujeongion/cold-tester

---


# /cold-tester — test your docs with an AI that has zero context

## Why this exists

You can't fairly test your own documentation. You already know what every step
*means*, so you fill the gaps automatically and the doc reads fine — to you. A real
newcomer doesn't have that context, and neither should the tester.

This skill runs the doc through an **isolated** model (a separate process, blank
slate, given only the document + a persona + a kickoff). It tries to actually do the
task, gate by gate, and every place it stalls, guesses, or writes "UNCLEAR — need to
confirm" is a place your doc is missing something.

Your main agent (this session) does **not** play the tester — it already knows too
much. It plays the **approver**: it reads each submission the cold tester produces and
rules pass / conditional / reject, then synthesizes which doc sections need fixing.

## How it works

```
Your main agent (orchestrator + approver)
  → passes the doc + persona + kickoff to an isolated model (the cold tester)
isolated model (zero context)
  → attempts each gate using ONLY the doc, flags every gap
  → returns submissions
Your main agent
  → rules pass / conditional / reject per gate, writes the report
```

## Setup

Any OpenAI-compatible endpoint. Use a **cheap** model — the tester should be naive,
not smart.

```bash
pip install openai pyyaml
export COLD_TESTER_API_KEY=sk-...                       # required
export COLD_TESTER_BASE_URL=https://api.openai.com/v1   # optional (default OpenAI)
export COLD_TESTER_MODEL=gpt-4o-mini                    # optional (or set in config)
```

Works with OpenAI, DeepSeek, Together, OpenRouter, or any local OpenAI-compatible
server — just point `COLD_TESTER_BASE_URL` at it.

## Run it

The script stays at its install path; `--base` points it at the repo under test, so
you never copy it into the target project.

Config file (recommended):

```bash
python3 ~/.claude/skills/cold-tester/cold_tester.py --config test.yaml --base /path/to/repo
```

Or inline:

```bash
python3 ~/.claude/skills/cold-tester/cold_tester.py \
  --base /path/to/repo \
  --docs README.md,CONTRIBUTING.md \
  --persona "First-time open-source contributor, has never seen this repo" \
  --task "Open your first pull request" \
  --kickoff "Set up the project locally and open a PR fixing a typo." \
  --gates "setup,find-an-issue,open-pr" \
  --output results/contributing-test.md
```

The cold tester reasons through the docs as a newcomer would; it doesn't execute
commands or touch the filesystem. The gaps it reports are real, the execution is
simulated — that's the reading-comprehension test you want.

### Config (YAML)

```yaml
test_name: "CONTRIBUTING guide — first PR"
model: gpt-4o-mini          # optional; overrides COLD_TESTER_MODEL
persona: |
  You are a competent developer who has never contributed to this project.
  You know general Git/GitHub but nothing about this repo's conventions.
task: "Open your first pull request"
kickoff: |
  Set up the project locally and open a small PR.
documents:
  - path: README.md
  - path: CONTRIBUTING.md
gates:
  - id: setup
    name: "Local setup"
    approver: "you"
    instruction: "Follow the setup steps and report exactly where you got stuck."
  - id: open-pr
    name: "Open the PR"
    approver: "you"
    instruction: "Describe the exact commands/clicks you'd run to open the PR."
```

## The approver's job (your main agent)

After the script writes the report, fill each gate's verdict:
- **pass** — a newcomer could complete this gate from the doc alone
- **conditional** — completable but with a guess/ambiguity worth fixing
- **reject** — the doc doesn't give enough to proceed

Then write the Summary: which sections caused stalls, and the specific doc edits that
would fix them. A gate the cold tester marked "UNCLEAR" but you'd have known how to do
is the highest-value finding — that's the curse-of-knowledge gap made visible.

## Good things to cold-read

Onboarding guides, CONTRIBUTING / setup docs, runbooks and SOPs, API quickstarts,
tutorials, any "follow these steps" document where the author's context silently fills
the gaps.