Minimal Answer Verification

Calibrate and verify minimal outputs for automatically-graded short answers. Use when the answer span appears verbatim in the supplied context, when a context offers both specific and generic phrasings, or when a yes/no claim shares a salient keyword with evidence but the actual outcome must be matched.

ruc-nlpir Updated 23 repo stars

File contents

  1. Emit the verbatim span immediately. When the answer token appears verbatim (and often repeatedly) in the supplied context, copy that minimal span and output it as the final answer at once. Do not invoke a skill or keep deliberating after it is found — delay or extended internal reasoning can leave the run with no answer output and score 0.

  2. Prefer the shortest, most general base noun. When the context offers both a specific phrase and a generic term, answer with the bare noun the question requires and strip descriptive modifiers the question does not itself require (e.g., "balloon", not "hot air balloon").

  3. Verify the semantic consequence, not the shared keyword. When a yes/no claim overlaps a passage on a salient phrase, do not let that shared term drive the polarity. Locate the passage stating the actual outcome and match the claim's consequence (dies vs. survives), not merely the triggering event the claim shares with the text.

ruc-nlpir/flashrsi commit b49f4377f9

Frequently asked questions

npx get-skill add ruc-nlpir/minimal-answer-verification