# Case Verification > Fail-closed verification of legal case citations against CourtListener. Detects fabricated cases, wrong pincites, and fabricated or misquoted language in briefs, motions, memos, and opinions. Use this skill whenever the user asks to verify citations, check quotes, catch AI hallucinations in legal writing, audit a brief or motion before filing, find errors in opposing counsel's filings, or review any legal document with case citations — even if they don't use the word "verify" (phrases like "check the cites in this motion", "find hallucinations", "is this quote real", "did they make this up", "review opposing counsel's brief", or "is this case real" all apply). US federal and state courts only. - Skill: `dave817/case-verification` (Agent Skill, multi-file: 6 files) - Install (CLI): `npx skillmds@latest add dave817/case-verification` - Raw SKILL.md: https://api.skillmd.com/api/skills/dave817/case-verification/raw - Safety review: pending - Works with: Claude Code, Claude.ai, OpenAI Codex - Category: Security - Author: dave817 (https://skillmd.com/u/dave817) - Updated: 2026-09-17 - Page: https://skillmd.com/skills/dave817/case-verification --- # case-verification Deterministic citation verification for US legal writing. Built against the Sullivan & Cromwell Schedule A of errors from *In re Prince Global Holdings Limited* (Bankr. S.D.N.Y. Apr. 18, 2026) as the benchmark: fabricated cases, pincite drift, and fabricated quotations — the exact failure modes of current AI legal drafting. ## What this skill does Given a brief, motion, memo, opinion, or any legal document, the skill runs a deterministic pipeline that: 1. Extracts every case citation with **eyecite** (Free Law Project parser) 2. Resolves each citation against **CourtListener** (5M+ US opinions, free API) 3. Fetches the full opinion text where the citation resolves 4. Checks whether any quoted language associated with the citation actually appears in that opinion, with normalization for quote style, whitespace, ellipses, and bracketed editorial insertions 5. Where Westlaw-style star-pagination (`*5`, `*10`) is present in the opinion, checks that the pincite in the brief matches where the quoted language actually appears 6. Produces a structured evidence ledger + a human-readable report **Verification is fail-closed.** A citation is marked `verified` only when the deterministic checks pass with evidence in hand. When the evidence isn't there, the output says `unresolved` or `not_found`, not "probably fine." ## What this skill does NOT do Be direct with the user about the scope: - It does **not** decide whether a cited case actually supports the proposition for which it is cited. That requires legal judgment and remains with the human reviewer. The report retrieves and displays the relevant passage so the human can decide — but the machine does not pretend to clear substantive support. - It does **not** check Shepard's / KeyCite treatment (overruled, abrogated, distinguished). Those citators are not freely available. - It does **not** cover non-US law. BVI, Cayman, UK, Canadian, Australian, and other foreign citations are flagged `unresolved` with a note. That is the honest answer; say so. - It does **not** verify statutes, regulations, legislative history, or secondary sources. When the user asks about any of the above, explain the scope limit rather than guessing. ## When to use this skill Use this skill, and call the verify.py script, whenever the user: - Uploads or pastes a brief, motion, memo, opinion, or legal draft and asks for any form of citation review, accuracy check, or hallucination audit - Asks whether a specific case citation is real - Asks whether a quote attributed to a case actually appears in the case - Asks to audit opposing counsel's filing for errors (adversary mode) - Asks to produce a pre-filing citation check (pre-flight mode) - Uses language like "check the cites", "find hallucinations", "is this real", "did they make this up", "verify this quote", "review before filing", "is this case on point" Do NOT answer these questions from memory. Always run the script. The entire point of the skill is that the model's confidence about whether a citation is real is not evidence. ## Two modes **Pre-flight mode** (default, `--mode preflight`): the user is verifying their own draft. Output lists each citation with its status, the specific error class if any, and a narrow suggested correction. Tone is that of a careful colleague pointing out what needs fixing. **Adversary mode** (`--mode adversary`): the user has received the adversary's document and wants to identify unverifiable citations to use in a response, a letter to the court, or a 28(j) filing. Output leads with a flagged-items section that contains finding-style framing plus a draftable paragraph for each flagged citation. The draftable paragraph is written to be usable, with citations to the retrieved opinion text as the evidentiary basis. Infer the mode from context when possible. If the user says "I'm writing this brief", it's pre-flight. If they say "opposing counsel filed this", it's adversary. When it's genuinely unclear, ask. ## Workflow Follow these steps in order. ### 1. Read the skill scripts Before doing anything else, view `scripts/verify.py` and note the CLI flags it exposes. The skill's logic lives in the scripts — do not re-implement any of it in prose. ### 2. Prepare the input The input is a brief as plain text or markdown. If the user uploads a PDF or Word file, convert it to plain text first (the document-reading skills handle this; ask the user to confirm the conversion preserved the citations). Save the text to a working file. ### 3. Check for an API token CourtListener works unauthenticated but is rate-limited to 100 lookups per day. For any real-world brief the user should set `COURTLISTENER_API_TOKEN` (free token, sign up at courtlistener.com). If the env var is not set, tell the user once at the start of the session; do not repeat the warning every run. ### 4. Run the verifier ```bash python3 scripts/verify.py --mode --format md --output report.md ``` Useful flags: - `--mode preflight` (default) or `--mode adversary` - `--format md` (default) or `--format json` or `--format both` - `--output ` — write to file instead of stdout - `--offline` — parse-only, no CourtListener calls. Use this for a dry-run when the network is unavailable or when the user wants to see what the tool will check without hitting the API. - `--token ` — override the env var ### 5. Present the report The script's own output is already the report. Do not rewrite it into prose or summarize it to the point of losing the per-citation detail — the user needs to see which specific paragraphs have which specific issues, with the retrieved evidence snippets and source URLs. That traceability is the whole point. For long reports, it is fine to give the user a one-paragraph summary at the top of your response followed by the full report (or a link to the output file if it was written to disk). ### 6. Respond to follow-ups Common follow-ups and how to handle them: - *"Can you re-check citation X?"* — re-run verify.py with a small input that contains just that citation. Do not answer from memory. - *"What does this case actually say about Y?"* — fetch the opinion text via the CourtListener URL that's already in the report, read it, and answer from the retrieved text with a direct source citation. This is legal research, not verification — be clear about the switch. - *"Write the response brief paragraph"* — in adversary mode the report already contains draftable paragraphs. Use those as the basis, tighten them for the specific context, but do not invent additional case citations. ## Output format The script outputs per-citation findings with **layered status** — each citation is checked on multiple dimensions and each dimension is reported separately. The top-level badge is a roll-up, and deployers must not treat it as a substitute for the layer detail. ### Layered status (shown in every finding) | Layer | What it checks | |---|---| | A. Authority resolution | Does the citation resolve in CourtListener? | | B. Source retrieval | Can we retrieve opinion text? | | C. Metadata consistency | Do year and case name match the brief? | | D. Quote verification | Does quoted language appear in the opinion? | | E. Pincite verification | Is the `*N` page consistent with where the quote appears? | | F. Proposition support | **ALWAYS `not_reviewed`** — engine does not judge | | G. Treatment / citator | **ALWAYS `not_reviewed`** — no KeyCite / Shepard's | Layers F and G being permanently `not_reviewed` is not a bug. The engine refuses to claim it has checked things it has not checked. See `GOVERNANCE.md` for the full honesty model. ### Top-level status vocabulary | Status | Meaning | |---|---| | `verified_quote_and_metadata` | Resolved; quote verified; metadata consistent. **Support NOT reviewed.** | | `resolved_no_quote_support_not_checked` | Resolved; no quote asserted. **Support NOT reviewed.** | | `pincite_error` | Case exists, quote is in it, but at a different `*N` page | | `quote_not_found` | Case exists, but the quoted language isn't in it | | `metadata_mismatch` | Resolved, but year or case name disagrees | | `not_found` | Did not resolve in CourtListener — potential hallucination | | `ambiguous` | Multiple candidate cases match the citation | | `unresolved` | Out of scope, or CourtListener couldn't be reached | Note that **no top-level status says bare `verified`**. Even in the best case, the label is `verified_quote_and_metadata` — a deliberate reminder that proposition support was not checked. Each finding includes: the citation as used, canonical record (if found), the quoted language (if any), the quote-check result, the pincite-check result, a retrieved evidence snippet, a source URL, provider class (`PUBLIC_FALLBACK` for the default engine), and — in pre-flight mode — a narrow suggested correction. ## Honesty rules These are non-negotiable: 1. **The engine never emits bare `verified`.** Even the best-case badge is `verified_quote_and_metadata`, to keep the reader aware that proposition support was not checked. 2. **Layers F and G are always `not_reviewed`.** Proposition support and treatment are outside engine scope. The layered status table must show this explicitly on every finding. 3. If CourtListener is not reachable or the opinion text is not retrievable, no quote can be called verified. The layers show `text_unavailable` and the top-level status is `unresolved`. 4. Non-US citations or reporters CourtListener doesn't cover are flagged `unresolved — out of scope` with a note. Do not attempt to verify foreign citations from model knowledge. 5. Every finding discloses its provider class (`PUBLIC_FALLBACK` for the default CourtListener engine). A firm deployment overriding the provider must update this disclosure. 6. If the user asks whether a case is still good law, or whether a case actually supports a proposition, say directly that the engine does not check these things and point them to `GOVERNANCE.md` or to a commercial citator for treatment. ## Files in this skill ``` case-verification/ ├── SKILL.md this file ├── README.md GitHub-facing overview ├── GOVERNANCE.md deployment policy for firms (status │ semantics, provider classes, materiality, │ filing-gate recommendations) ├── scripts/ │ ├── verify.py main entry point │ ├── parse.py eyecite-based extraction of verification units │ ├── cl_client.py CourtListener API client (stdlib only) │ └── text_utils.py quote normalization + star-pagination └── tests/ ├── sample_brief.txt S&C-style fixture with known errors └── test_pipeline.py integration tests including adversarial cases ``` The only external dependency is `eyecite` (install with `pip install eyecite`). The CourtListener client uses Python stdlib only, no `requests`.