LLM As Judge Designer

Iterate-stage skill: turns existing eval criteria into an LLM judge prompt with anchors and few-shot calibration cases — every rubric point carrying both a pass and a fail exemplar. Use when criteria exist and the judge needs building — 'write the judge prompt for these criteria', 'turn this rubric into an LLM judge', 'judge prompt plus calibration cases' — or when /pm routes such a request here. Do NOT use to build the whole eval from a spec (eval-engine), to audit an existing judge against human labels (judge-calibration-auditor), to execute scoring over outputs, or for judge-reliability knowledge questions.

Abhillashjadhav Updated 9 repo stars

File contents

Abhillashjadhav/PM-agent-OS commit 3d33c36238

Frequently asked questions

npx get-skill add abhillashjadhav/llm-as-judge-designer