# Citation Verification

> Use this skill to add a post-generation pass that verifies every citation in the LLM's answer actually appears in (and is supported by) the cited chunk. Reach for it when the faithfulness score is low even though retrieval is healthy, or when invented citations are surfacing as a failure category.

- Skill: `allanwessels/citation-verification` (Agent Skill)
- Install (CLI): `npx skillmds@latest add allanwessels/citation-verification`
- Raw SKILL.md: https://api.skillmd.com/api/skills/allanwessels/citation-verification/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: AllanWessels (https://skillmd.com/u/allanwessels)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/allanwessels/citation-verification

---


# Citation Verification

## When to use

Add a verification pass when `/reports/latest.json` shows:
- High retrieval recall (>= 0.7 avg) but low faithfulness (< 0.6 avg)
- Failures in the "invented citation" category, surfaced by the judge

Don't add it if faithfulness is already high — it's an expensive pass
that you should only enable when the evidence demands it.

## How it works

After the LLM generates an answer with chunk-numbered citations, run
a separate verification pass:

1. Parse the answer into (claim, cited-chunk-id) pairs. Most
   citation formats like `... [3] ...` are easy to regex.

2. For each (claim, chunk) pair, call a fast LLM with a fixed prompt:
   ```
   Claim: {claim}
   Chunk: {chunk_text}

   Does the chunk directly support this claim? Reply with one word:
   YES, NO, or PARTIAL.
   ```

3. If all pairs return YES, the answer is verified. If any return NO,
   the answer has at least one fabricated citation. PARTIAL is a soft
   warning.

## What to do on failure

Three reasonable strategies depending on stakes:

- **Retry**: regenerate the answer with the same chunks but stronger
  grounding instructions. Usually fixes single-claim issues.
- **Downgrade**: replace the offending claim with "I'm not sure"
  or drop it from the answer. Keeps the rest of the answer usable.
- **Refuse**: return an honest "I don't have enough information"
  response. Highest-trust, lowest-coverage option.

For most consumer-facing systems, retry once then downgrade. For
high-stakes (medical, legal, financial) systems, refuse rather than
serve unverified content.

## Implementation requirements

Your generation prompt needs to use a citation format the verifier
can parse:

```
Use citations in the format [N] where N is the chunk number.
Place the citation immediately after the claim it supports.
Every factual claim must have a citation.
```

Then your verifier parses on `\[(\d+)\]` and matches each claim
(sentence containing the citation) to the chunk by number.

## Cost considerations

A verification pass adds one LLM call per cited claim. For a
typical answer with 3-5 citations, that's 3-5 small LLM calls
added to each query. The calls are cheap (small context, single
word output) but they're real latency. Budget for them.

Optimization: many claims share chunks. Group claims by chunk and
verify in batch — one call per (chunk, set-of-claims) pair instead
of per claim.

## When NOT to use

- Low-stakes use cases where occasional fabrication is acceptable
- Cases where the user can verify the answer themselves
- Early iteration — focus on retrieval and prompt-level grounding
  before adding a verification pass