Aipm LLM Judge

Writes an LLM-as-a-judge prompt for one failure type and the check that proves it, a labelled agreement set, agreement with human labels on a held-out split, and bias checks for position, length and self-preference. Use for "run aipm-llm-judge", "LLM as a judge", "judge prompt", "automate our eval grading", "can we trust the LLM grader", "model-graded evals", "validate our judge against humans", part of the Claude for AI Product Managers Pack by 90n.

polar-bear-org Updated 38 repo stars

File contents

polar-bear-org/claude-skills commit 8cbd5f5583

Frequently asked questions

npx get-skill add polar-bear-org/aipm-llm-judge