Umbrela Verify

Use when validating umbrela judge outputs — checks label range (0–3), qid/docid completeness, result_status consistency, backend metadata, and JSONL integrity. Wraps `umbrela validate` plus custom assertions. Use after running judge or evaluate to verify output correctness.

castorini Updated

File contents

castorini/umbrela commit 446de58fb6

Frequently asked questions

npx get-skill add castorini/umbrela-verify