How do I install the Testing CLI skill?

Run npx skillmds@latest add muratcankoylan/testing-cli in your terminal (requires Node.js), paste this page's agent-chat prompt into Claude, Cursor, or any MCP-connected agent, or download the SKILL.md file and copy it into your agent's skills directory.

Is Testing CLI safe to use?

This skill has not completed SkillMD's automated safety review yet. SkillMD never runs a skill's scripts for you; review the SKILL.md before installing.

Which AI agents work with Testing CLI?

This skill is tagged as working with Claude Code, Claude.ai, OpenAI Codex. SKILL.md is an open format, so most agents that read a skills directory can load it too.

Is Testing CLI free to use?

Yes. Installing skills from SkillMD is free, and the skill stays under its author's original license.

Who published Testing CLI?

Murat Can Koylan (@muratcankoylan) published this skill. Their other Agent Skills are listed on their SkillMD profile.

Testing CLI by muratcankoylan — Agent Skill

Name: Testing CLI
Author: Murat Can Koylan

File contents

Testing the peoplereadme CLI

All testing is shell-only (no GUI). Use a clean clone + uv sync, then uv run peoplereadme ....

Basic checks

uv run ruff check src tests and uv run pytest -q (all tests offline, no API keys needed).
uv run peoplereadme validate must exit 0 (riley-walz must stay schema-valid).

Synthetic persona fixture (for ingest/trace/eval testing)

uv run peoplereadme init me --class self
Build a synthetic X export zip: data/account.js + data/tweets.js with window.YTD.account.part0 = [...] / window.YTD.tweets.part0 = [...] prefixes; tweets are {"tweet": {"id_str", "full_text", "created_at": "%a %b %d %H:%M:%S +0000 %Y", ...}}. Self-threads (replies to own tweets via in_reply_to_status_id_str) produce traces with reconstruction_quality 0.8; standalone posts get 0.2 (both pairwise-eligible; the harness excludes < 0.2). Use ~40 tweets so the 15% test split has ≥5 eligible traces.
uv run peoplereadme ingest me --source x-archive=archive.zip then uv run peoplereadme trace me.

Harness (M2) live testing

Requires OPENAI_API_KEY (available as a saved Devin secret). Use --model openai/gpt-4o-mini and small --n-pairs (e.g. 6) to keep cost < $1; a 6-pair run with baseline takes ~70s.
uv run peoplereadme eval me --model openai/gpt-4o-mini --n-pairs 6 --seed 7 writes personas/me/evals/fidelity/<date>.json, auto-creates evals/rubrics/v1.json, and exports a blind calibration batch to evals/fidelity/calibration/<date>.jsonl (+ private <date>.answers.json key).
Judge calls are disk-cached in ~/.peoplereadme/cache/llm (override root with PEOPLEREADME_CACHE); delete to force fresh judging.
Calibration roundtrip: fill human_pick ("A"/"B") in the blind JSONL, then uv run peoplereadme calibrate me --batch <date> --ratings rated.jsonl. Expect exit 1 with "INVALID report card" whenever kappa < 0.4 or fewer than 50 rated pairs — that hard floor firing is correct behavior, not a bug.
Expected diagnostics gotcha: with a skeleton persona package (no real context files) the judge often scores below chance and generated outputs are far longer than real tweets — check diagnostics in fidelity.json (position_bias, mean_real_len, mean_generated_len).

Failure modes (should all be clean errors, exit 1, no tracebacks)

eval on a persona without traces/traces.jsonl
calibrate with a missing batch/answers key
eval with an invalid API key (LMError path)

---name: testing-clidescription: How to test the peoplereadme CLI end-to-end, including the fidelity harness (eval/calibrate) with a synthetic persona fixture and live LLM calls.---# Testing the peoplereadme CLIAll testing is shell-only (no GUI). Use a clean clone + `uv sync`, then `uv run peoplereadme ...`.## Basic checks- `uv run ruff check src tests` and `uv run pytest -q` (all tests offline, no API keys needed).- `uv run peoplereadme validate` must exit 0 (riley-walz must stay schema-valid).## Synthetic persona fixture (for ingest/trace/eval testing)1. `uv run peoplereadme init me --class self`2. Build a synthetic X export zip: `data/account.js` + `data/tweets.js` with `window.YTD.account.part0 = [...]` / `window.YTD.tweets.part0 = [...]` prefixes; tweets are `{"tweet": {"id_str", "full_text", "created_at": "%a %b %d %H:%M:%S +0000 %Y", ...}}`. Self-threads (replies to own tweets via `in_reply_to_status_id_str`) produce traces with reconstruction_quality 0.8; standalone posts get 0.2 (both pairwise-eligible; the harness excludes < 0.2). Use ~40 tweets so the 15% test split has ≥5 eligible traces.3. `uv run peoplereadme ingest me --source x-archive=archive.zip` then `uv run peoplereadme trace me`.## Harness (M2) live testing- Requires `OPENAI_API_KEY` (available as a saved Devin secret). Use `--model openai/gpt-4o-mini` and small `--n-pairs` (e.g. 6) to keep cost < $1; a 6-pair run with baseline takes ~70s.- `uv run peoplereadme eval me --model openai/gpt-4o-mini --n-pairs 6 --seed 7` writes `personas/me/evals/fidelity/<date>.json`, auto-creates `evals/rubrics/v1.json`, and exports a blind calibration batch to `evals/fidelity/calibration/<date>.jsonl` (+ private `<date>.answers.json` key).- Judge calls are disk-cached in `~/.peoplereadme/cache/llm` (override root with `PEOPLEREADME_CACHE`); delete to force fresh judging.- Calibration roundtrip: fill `human_pick` ("A"/"B") in the blind JSONL, then `uv run peoplereadme calibrate me --batch <date> --ratings rated.jsonl`. Expect exit 1 with "INVALID report card" whenever kappa < 0.4 or fewer than 50 rated pairs — that hard floor firing is correct behavior, not a bug.- Expected diagnostics gotcha: with a skeleton persona package (no real context files) the judge often scores below chance and generated outputs are far longer than real tweets — check `diagnostics` in fidelity.json (position_bias, mean_real_len, mean_generated_len).## Failure modes (should all be clean errors, exit 1, no tracebacks)- eval on a persona without `traces/traces.jsonl`- calibrate with a missing batch/answers key- eval with an invalid API key (LMError path)

Frequently asked questions

Testing CLI