WanLanglin jev-triage
The value here is not the labels — you can label things yourself. The value is that the obvious cases cost nothing, so you only spend real reasoning where it is needed. Read the confidence caveat below before you trust the split.
Use it
python3 "$CLAUDE_SKILL_DIR/../../scripts/triage.py" \
--question "Is this review finding a real defect?" \
--label real="a genuine bug that would cause wrong behaviour or a crash" \
--label style="naming, formatting, or preference only" \
--label wrong="the finding misreads the code and is not a defect" \
--escalate-below 0.9 \
--items findings.txt
Multi-line items (stack traces, diff hunks, findings with code):
python3 .../triage.py --sep $'\n---\n' --items hunks.txt \
--question "How risky is applying this hunk automatically?" \
--label safe="isolated, easy to revert, well covered by tests" \
--label review="touches shared state or public API" \
--label danger="security, auth, migrations, or data loss"
Add shared background once with --context — it is billed once for the whole
batch, not per item, so be generous with it.
The part that matters
Output splits into decided buckets and an escalate list. The escalate list is the whole point: those are the items where a confident-looking answer would have been wrong. Send exactly those to a subagent or handle them yourself, and accept the rest.
--escalate-below defaults to 0.90, and that number is measured, not
chosen for feel. On a 500-decision labelled task, choice confidence was close
to uninformative between 0.4 and 0.9 — accuracy wandered between 26% and 50%
across five bins — and only separated above 0.9, where it reached 80.9%
(ECE 0.226).
| escalate below | automated | accuracy of that automated set |
|---|---|---|
| 0.7 | 77% | 65% — one in three wrong |
| 0.9 | 41% | 81% |
So: do not lower this without measuring on your own data. choice
confidence is meaningfully weaker than the noul probability jev-shortlist
uses. Even at 0.9 you are accepting roughly one error in five, which is fine for
triage and not fine for anything irreversible. Full curve and caveats:
references/calibration.md.
Cost
Up to 256 items per call, content billed once. A 256-item batch runs well under a cent and under a second. Compare that with one subagent per item.
Setup
export TYPESAFE_API_KEY=... # from typesafe.ai