Answer Source Form Verification

Determine the authoritative source, answer type, and exact surface form before committing an exact-match short answer. Use when the retrieved context interleaves a speaker's guess with an 'A:' answer, when the prompt labelled 'Question' is a bare declarative statement with an unresolved pronoun, or when the prompt's 'one of these [plural]' phrasing dictates a singular noun with article.

ruc-nlpir Updated 23 repo stars

File contents

  1. Trust the authoritative marker, not the plausible guess. In a Jeopardy!/Q&A transcript, a named speaker's response (e.g., "CORDRAY: What is gravity?") is often a WRONG guess; the real answer follows the "A:" marker. Extract the "A:" value, and discount conflicting forum posts when independent sources agree.

  2. Resolve the prompt's actual answer type. When the "Question" is a bare declarative statement with an unresolved pronoun ("his", "her", "their") and no question word, it is not a True/False claim. The expected short answer is the named entity the pronoun refers to — resolve it against the context and emit that entity's name.

  3. Match grammatical number and article to the prompt's phrasing. A blank introduced by "one of these [plural category]" ("...one of the ... holes") wants the singular count noun with its indefinite article ("a sinkhole"), not a bare plural ("sinkholes").

  4. Before committing, check three things: what answer type the prompt requests (entity vs. boolean), which context region is authoritative vs. distractor, and what exact surface form (article/number) the key expects. Do not let the most salient or plausible token override these.

ruc-nlpir/flashrsi commit f062a82f55

Frequently asked questions

npx get-skill add ruc-nlpir/answer-source-form-verification