Mk Rubric

Rubric Library API

Discovery, composition, and validation for the rubric library at .codex/rubrics/.

Path convention: Commands below assume cwd is $(git rev-parse --show-toplevel) (project root). Prefix paths with "$(git rev-parse --show-toplevel)/" when invoking from subdirectories.

Consumed by evaluator agent + mk:evaluate skill (Phase 3, shipped 260408). The dedicated evaluator sub-task (.codex/agents/evaluator.md) drives the running build via active verification and grades it against rubrics composed from this library. The mk:evaluate skill (.agents/skills/evaluate/) is the orchestration shell. This skill (mk:rubric) remains independently invokable via the rubric skill <subcommand> for manual rubric inspection, validation, or composition outside the evaluator workflow.

When to Use

Activate when:

• User runs the rubric skill [subcommand]
• An evaluator sub-task needs to load rubrics for grading
• Sprint-contract negotiation references a rubric by path
• CI validates rubric schema conformance after edits

Subcommands

Usage Examples

# List all rubrics
.agents/skills/rubric/scripts/load-rubric.sh --list

# Load one rubric for the evaluator
.agents/skills/rubric/scripts/load-rubric.sh design-quality

# Compose a preset (returns all member rubrics + weight table)
.agents/skills/rubric/scripts/load-rubric.sh --preset frontend-app

# Validate every rubric file in the library
.agents/skills/rubric/scripts/validate-rubric.sh

Output Schema

load <name> output

## Rubric: {name} (weight: {weight_default}, hard_fail: {threshold})

{Intent paragraph}

### Criteria
{bullets}

### Grading
{table}

### Anti-patterns
{bullets}

### Few-Shot Examples
{PASS + FAIL examples, balanced — section name matches schema.md and the actual rubric files}

compose <preset> output

## Composition: {preset-name}

| Rubric | Weight | Hard-Fail Threshold |
|---|---|---|
| ... | ... | ... |

(All member rubrics inlined below)

Schema

Rubrics MUST conform to .codex/rubrics/schema.md. The validator enforces:

• Required frontmatter fields (name, version, weight_default, applies_to, hard_fail_threshold)
• Required sections in order
• ≥1 PASS + ≥1 FAIL anchor example, balanced (±1)
• File ≤200 lines

Composition presets MUST have all weights summing to 1.0 ±0.01.

Calibration

Few-shot anchor examples MUST be added per the rules in references/calibration-guide.md:

• Balanced PASS/FAIL counts (positive bias mitigation)
• Randomized presentation order (position bias mitigation)
• Drawn from real prior reviews where possible (synthetic examples only as backup)

Gotchas

• Adding a new rubric: drop a .md file into .codex/rubrics/, run validate-rubric.sh, register weight in any preset that should include it.
• Weight drift: if you change a rubric's weight_default, all presets that reference it must be re-checked for sum=1.0.
• Anti-patterns are FIXED: they trigger FAIL regardless of the surrounding criteria. Don't add subjective ones.
• Don't load all rubrics: the evaluator should load only the relevant preset, not the whole library — context efficiency.

References

Start

For interactive use, run scripts/load-rubric.sh --list to discover available rubrics. For evaluator-internal use, the evaluator agent (via mk:evaluate) loads compose <preset> automatically based on the target type detected by mk:evaluate/scripts/run-evaluator.sh. Sprint-contract-based filtering is gated until Phase 4 ships (step-01 of mk:evaluate stub-guards on the contract_schema_version: marker).