# Nuggetindex

> Use when a RAG pipeline or knowledge base returns stale, outdated, or conflicting facts, or when the user mentions temporal drift, fact governance, contested answers, time-aware retrieval, fact freshness, entity renames (Twitter/X), or nuggetindex. Audits retrieved passages for stale and contested facts, maintains a governed time-aware fact ledger, and escalates to the nuggetindex pip package at corpus scale. Zero install for the common case.

- Skill: `searchsim-org/nuggetindex` (Agent Skill, multi-file: 5 files)
- Install (CLI): `npx skillmds@latest add searchsim-org/nuggetindex`
- Raw SKILL.md: https://api.skillmd.com/api/skills/searchsim-org/nuggetindex/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: DevOps & Infra
- License: MIT
- Author: searchsim-org (https://skillmd.com/u/searchsim-org)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/searchsim-org/nuggetindex

---


# NuggetIndex: time-aware, conflict-aware fact governance

You apply the NuggetIndex method (Zerhoudi et al., SIGIR '26): represent knowledge as
atomic facts ("nuggets") with explicit temporal validity, detect when sources disagree,
and **surface conflicts instead of silently picking a winner**.

## Core model

A **nugget** is one atomic fact:

- **Triple**: `(subject, predicate, object)` + the original supporting text span
- **Validity**: interval `[start, end)` — `end = null` means "still true as far as we know"
- **Lifecycle**: `active` (current) | `deprecated` (obsolete) | `contested` (sources disagree)
- **Provenance**: which source(s) assert it

Three failure modes this prevents:

1. **Temporal staleness** — answering "Who is Apple's CEO?" with Steve Jobs because an old
   passage ranked highest. Every fact carries `[start, end)`; check it against query time.
2. **Silent conflicts** — two sources claim different values for the same single-valued
   attribute; naive RAG picks whichever embeds better. Mark both `contested` and tell the user.
3. **Rename drift** — Twitter→X, Facebook→Meta. Track renames as facts (`renamedTo`) so both
   surface forms resolve to one entity.

## Decision ladder — cheapest rung that fits

### Rung 1: In-context audit (most common — no files, no install)

The user shows you retrieved passages, an answer that looks outdated, or asks "is my RAG
returning stale facts?":

1. Establish the **query time** (default: today; the user may ask about a past time).
2. Extract the fact triples relevant to the query from each passage, with any temporal
   cues — follow [references/extraction.md](references/extraction.md).
3. Infer each fact's validity interval from its temporal cues (rules in extraction.md).
4. Group facts by key `(subject, predicate)`. Apply the conflict rules in
   [references/conflict-rules.md](references/conflict-rules.md): single-valued predicate +
   overlapping validity + different objects ⇒ **contested**.
5. Report per passage: `consistent` / `stale` (validity ended before query time) /
   `contested` (disagrees with another passage), with the evidence spans.

Never resolve a contest yourself when evidence is symmetric — show both claims, their
sources, and their intervals, and let the user adjudicate.

### Rung 2: Project fact ledger (ongoing work, small working set)

The user wants governed facts across a session or project (docs, research notes, a small
knowledge base): maintain `nuggets.jsonl` — one nugget per line in the exact schema of
[references/ledger-format.md](references/ledger-format.md). The format is identical to the
pip package's `Nugget` JSON, so the ledger imports into a real index later with no migration.

On every append: extract → canonicalize entity names (resolve known renames/aliases to one
canonical form) → infer validity → dedup (same subject|predicate|object|start|scope = same
fact; merge provenance) → run conflict detection against existing ledger entries and update
lifecycle states. Report any new `contested` keys to the user immediately.

### Rung 3: Escalate to the pip package (scale, persistence, serving)

Switch when any of these hold: corpus beyond ~50 documents, need for persistent hybrid
BM25+dense retrieval, an HTTP sidecar for a production RAG stack, LangChain/LlamaIndex/
Haystack integration, or reproducible evaluation.

```bash
pip install nuggetindex        # extras: [openai] [anthropic] [dense] [serve] [all]
nuggetindex build ./corpus --db index.db
nuggetindex query "CEO Apple" --db index.db --time 2010-06-01
```

Full CLI/API surface, extras, and a hand-ledger import recipe:
[references/package.md](references/package.md). LLM-backed extraction needs an API key
(`OPENAI_API_KEY` / `ANTHROPIC_API_KEY` / `GOOGLE_API_KEY`, or local Ollama); the
rule-based extractor works with none. If no key is available and rule-based extraction is
too weak for the corpus, say so — do not fabricate extraction results.

## Rules that always apply

- **Timezone-aware UTC datetimes everywhere.** Intervals are half-open `[start, end)`.
- **Query-time semantics**: a fact answers a query at time `t` iff its validity contains `t`
  and its status is `active` or `contested` — never `deprecated`.
- **Surface, don't hide**: contested facts are presented as disputes with provenance, e.g.
  `Apple CEO: Tim Cook (2014–present) [DISPUTED: source B claims John Sculley (overlapping)]`.
- **Be literal in extraction**: no invented facts, no paraphrased evidence spans, omit facts
  you are not confident about.
- **Unknown predicates default to multi-valued** (conservative — avoids false conflicts).
  Only the single-valued predicates listed in conflict-rules.md trigger contests by default.