Skill Autobench

Author an eval for an existing skill from its REAL usage history, not its spec. Mine invocations from the brain's conversation archive (conversations/) and per-harness session transcripts — a user correction after an invocation is the gold signal — then synthesize an eval_contract plus 4-8 replayable cases with honesty labels (SPEC-DERIVED vs HISTORY-IMPLIED) and stage the result at skills/{name}/eval/autobench-{date}.md as PENDING-HUMAN-APPROVAL. Never rewrites SKILL.md. Ships two guard companions: panel integrity (multi-model judging must prove each provider actually responded) and the fail-improve taxonomy (logged LLM-fallback cases convert to deterministic code over time).

praxstack Updated 1 repo stars

File contents

praxstack/skills-and-personas commit b8bb4edbef

Frequently asked questions

npx get-skill add praxstack/skill-autobench