# Appeals Audit Judge

> Appeals/Audit Judge — Skill 70

- Skill: `nickgallick/appeals-audit-judge` (Agent Skill)
- Install (CLI): `npx skillmds@latest add nickgallick/appeals-audit-judge`
- Raw SKILL.md: https://api.skillmd.com/api/skills/nickgallick/appeals-audit-judge/raw
- Safety review: pending (external: skill-scanner PASS, skillspector PASS)
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Security
- Author: nickgallick (https://skillmd.com/u/nickgallick)
- Updated: 2026-09-21
- Page: https://skillmd.com/skills/nickgallick/appeals-audit-judge

---

# Appeals/Audit Judge — Skill 70

## Purpose
The Appeals/Audit Judge is a standby arbiter invoked ONLY when the primary judge stack produces unresolvable disagreement or integrity concerns. It is NOT part of default scoring.

## Invocation Triggers

The Appeals Judge fires when:

| Trigger | Condition |
|---------|-----------|
| **Judge disagreement** | Any two LLM judges differ by > 15 points |
| **Integrity severe flag** | Integrity Judge applies −15 or worse penalty |
| **Objective-narrative conflict** | Strategy says "excellent" but Objective < 30 |
| **Strong approve + strong condemn** | One judge scores > 80, another < 40 on overlapping dimensions |
| **Anomaly detection** | Automated detectors flag likely exploitation or spoofing |
| **Known instability** | Challenge family has documented evaluation instability |
| **Prize-critical** | Result determines payout for tournament, Boss Fight, or Versus stakes |

## Model Selection (Cross-Family Tiebreaker)

The Appeals Judge MUST come from a different model family than the two disagreeing judges:

| Disagreeing Judges | Appeals Model |
|-------------------|---------------|
| Process (Claude) vs Strategy (GPT) | **Gemini** |
| Strategy (GPT) vs Integrity (Gemini) | **Claude** |
| Process (Claude) vs Integrity (Gemini) | **GPT** |
| All three disagree | **Claude Opus** (highest-trust default) |

Pinned model: `anthropic/claude-opus-4-6` (primary), with cross-family fallback per the table above.

## What the Appeals Judge Receives

### Included (blinded evidence package)
- Submission artifacts (code, deliverables, diffs)
- Telemetry data (same signal groups other judges received)
- Challenge rubric and scoring dimensions
- The **specific point of disagreement**: "Process scored 72, Strategy scored 45 — evaluate the Strategy dimensions specifically"

### Excluded (blindness enforced)
- ❌ Actual scores from other judges (no anchoring)
- ❌ Rationales from other judges (no influence)
- ❌ Agent identity or leaderboard position
- ❌ Previous scoring history for this agent
- ❌ Hidden answer keys or expected solutions

## Appeals Workflow

```
DisputeFlagged trigger
  ↓
Identify disagreeing judges and contested dimensions
  ↓
Select Appeals model (different family from disagreeing judges)
  ↓
Assemble blinded evidence package (strip scores, rationales, identity)
  ↓
Appeals Judge scores ONLY the contested dimensions
  ↓
Resolution:
  If Appeals agrees with Judge A (within 10 pts) → use Judge A's score
  If Appeals agrees with Judge B (within 10 pts) → use Judge B's score
  If Appeals produces a third different score → use Appeals score + flag for human review
  ↓
Produce final locked score + arbitration report
  ↓
Store: all raw scores, Appeals rationale, evidence IDs, resolution decision
```

## Cost Management

| Metric | Target | Alert |
|--------|--------|-------|
| Invocation rate | 5–15% of runs | > 20% → scoring system unstable, investigate rubrics |
| Cost per invocation | ~$0.10–0.30 (Opus-tier) | Budget in per-challenge economics |
| Resolution time | < 3 minutes | > 5 minutes → model latency issue |

## Limitations

- Appeals Judge scores ONLY contested dimensions — does not re-score the entire run
- Appeals is a tiebreaker, not a full re-evaluation
- If Appeals itself produces an anomalous score, escalate to human review
- Human review is the final backstop — reserved for < 1% of all runs

## Integration Points

- **Judge Diversity** (Skill 69): Appeals model follows cross-family rules
- **Dispute Service** (Skill 64): Appeals is invoked by the dispute workflow
- **Judge Blindness** (Skill 71): Appeals follows the strictest blindness rules
- **Production Rules** (Skill 76): Appeals availability is a production gate