# Testing CLI

> How to test the peoplereadme CLI end-to-end, including the fidelity harness (eval/calibrate) with a synthetic persona fixture and live LLM calls.

- Skill: `muratcankoylan/testing-cli` (Agent Skill)
- Install (CLI): `npx skillmds@latest add muratcankoylan/testing-cli`
- Raw SKILL.md: https://api.skillmd.com/api/skills/muratcankoylan/testing-cli/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: Murat Can Koylan (https://skillmd.com/u/muratcankoylan)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/muratcankoylan/testing-cli

---


# Testing the peoplereadme CLI

All testing is shell-only (no GUI). Use a clean clone + `uv sync`, then `uv run peoplereadme ...`.

## Basic checks
- `uv run ruff check src tests` and `uv run pytest -q` (all tests offline, no API keys needed).
- `uv run peoplereadme validate` must exit 0 (riley-walz must stay schema-valid).

## Synthetic persona fixture (for ingest/trace/eval testing)
1. `uv run peoplereadme init me --class self`
2. Build a synthetic X export zip: `data/account.js` + `data/tweets.js` with `window.YTD.account.part0 = [...]` / `window.YTD.tweets.part0 = [...]` prefixes; tweets are `{"tweet": {"id_str", "full_text", "created_at": "%a %b %d %H:%M:%S +0000 %Y", ...}}`. Self-threads (replies to own tweets via `in_reply_to_status_id_str`) produce traces with reconstruction_quality 0.8; standalone posts get 0.2 (both pairwise-eligible; the harness excludes < 0.2). Use ~40 tweets so the 15% test split has ≥5 eligible traces.
3. `uv run peoplereadme ingest me --source x-archive=archive.zip` then `uv run peoplereadme trace me`.

## Harness (M2) live testing
- Requires `OPENAI_API_KEY` (available as a saved Devin secret). Use `--model openai/gpt-4o-mini` and small `--n-pairs` (e.g. 6) to keep cost < $1; a 6-pair run with baseline takes ~70s.
- `uv run peoplereadme eval me --model openai/gpt-4o-mini --n-pairs 6 --seed 7` writes `personas/me/evals/fidelity/<date>.json`, auto-creates `evals/rubrics/v1.json`, and exports a blind calibration batch to `evals/fidelity/calibration/<date>.jsonl` (+ private `<date>.answers.json` key).
- Judge calls are disk-cached in `~/.peoplereadme/cache/llm` (override root with `PEOPLEREADME_CACHE`); delete to force fresh judging.
- Calibration roundtrip: fill `human_pick` ("A"/"B") in the blind JSONL, then `uv run peoplereadme calibrate me --batch <date> --ratings rated.jsonl`. Expect exit 1 with "INVALID report card" whenever kappa < 0.4 or fewer than 50 rated pairs — that hard floor firing is correct behavior, not a bug.
- Expected diagnostics gotcha: with a skeleton persona package (no real context files) the judge often scores below chance and generated outputs are far longer than real tweets — check `diagnostics` in fidelity.json (position_bias, mean_real_len, mean_generated_len).

## Failure modes (should all be clean errors, exit 1, no tracebacks)
- eval on a persona without `traces/traces.jsonl`
- calibrate with a missing batch/answers key
- eval with an invalid API key (LMError path)