# Swe Rm Verifier

> Improve SWE agent RL via execution-free reward models optimized for three metrics beyond TTS: discriminative ability (AUC) and calibration (ECE). Shows TTS alone insufficient—models must distinguish correct/incorrect trajectories and align confidence with actual correctness—improving SWE-Bench Verified by 7-10 points with 30B MoE verifier.

- Skill: `adu2021/swe-rm-verifier` (Agent Skill)
- Install (CLI): `npx skillmds@latest add adu2021/swe-rm-verifier`
- Raw SKILL.md: https://api.skillmd.com/api/skills/adu2021/swe-rm-verifier/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- License: MIT
- Author: adu2021 (https://skillmd.com/u/adu2021)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/adu2021/swe-rm-verifier

---


## Overview

SWE-RM shows that reward model evaluation requires three complementary metrics.

## Core Technique

**Three-Metric Evaluation:**

```python
# Not just TTS (top-1 ranking)
tts_score = rank_best_solution_first(predictions)

# Also discriminative ability
auc_score = compute_auc(correct_vs_incorrect)

# And calibration
ece_score = expected_calibration_error(confidence, accuracy)
```

## When to Use

Use when: SWE agent training, RL reward modeling, importance of calibration.

## References

- Discriminative ability vs ranking metrics
- Calibration error measurement
- Multi-objective reward model design