Knowledge Graph

Knowledge Graph: Four Prompts, One Schema

Structured outputs replace four separate trained systems with four prompts that share one schema: the schema is the only training data. Adapting to a new domain means changing the schema and prompt, then evaluating the result.

Say it while working:

"No NER model to train — the schema is the only training data; tightening the type Literal is our retraining step."

When a graph is the right tool

Route before you build. A knowledge graph pays off only when facts must be chained across documents, shared across agents, or checked with provenance.

Say the routing-out sentence when it applies:

"This is single-hop retrieval over one collection — RAG, not a graph; stopping here."

The four-prompt pipeline

Use the extraction tier for schema-constrained volume and the generation tier for judgment over conflicting evidence. Resolve current models through the host or collection routing; do not pin model ids in this workflow.

Full schemas, prompt text, and code: references/pipeline.md.

1. Extraction

One call per document replaces both NER and relation classification: a list of typed entities plus subject-predicate-object triples. Every prompt guideline exists to fix a specific failure mode — keep all five:

• extract only entities central to the document → recall control
• one-sentence description grounded in that document → the disambiguation signal resolution depends on
• short verb phrases as predicates → constrained predicate vocabulary
• every relation connects two extracted entities → no orphaned edges
• a verbatim span quoted for each relation → checkable provenance

2. Entity resolution

Cluster surface forms into canonical nodes, one entity type at a time. The one-line descriptions are the signal: they merge "Edwin Aldrin" with "Buzz Aldrin" — zero character overlap, where string similarity fails outright — and keep "Armstrong, first person on the Moon" apart from "Armstrong, jazz trumpeter". Monitor the two failure modes: silent loss (a name left out of every cluster) and over-merging (folding "Gemini 12" into "Project Gemini").

3. Entity summarization

Pool every mention plus the graph neighborhood into a cross-document profile — hub nodes only, degree ≥ 3. Below that the single-document description suffices; this is the expensive stage, so apply it selectively.

4. Graph-grounded querying: local and global

Every question forks first on whether it has a seed in the graph to start from.

• Local search — the question names an entity. Link the mention through the alias map to its canonical node, serialize the k-hop neighborhood as (subject) --[predicate]--> (object) triples, reason over them. k=2 is the sweet spot: it captures the chains that make the graph valuable without flooding the context. When the alias map has no match, the answer is "the graph contains no entity matching X" — not a fallback to pretraining.
• Global search — the question names no entity and asks for themes, patterns, recurrence, or coverage. There is no seed to walk from, and the answer is a property of the whole graph. Cluster the graph into communities, summarize each once, and map-reduce the question over those reports: references/global-search.md.

Say which fork you took:

"This question names no entity — global search over community reports, not a k-hop neighborhood."

Global search adds no fifth model to the pipeline: a community report is prompt 3's move at a coarser granularity — summarize a cluster instead of a node — and runs on the same judgment tier.

Either way the answer is grounded: it cites specific edges and their evidence spans, and states what the graph does not contain. An ungrounded answer draws on pretraining and cannot be verified. On a private corpus, only grounded answers work at all.

Precision over recall

A wrong entity is worse than a missing one: it spawns wrong relations that propagate through every multi-hop chain that touches it, while a missing entity is a visible gap. Tune extraction toward precision, and improve with the harness loop: change the prompt, rerun the scorer against the gold set, watch F1 move. Score through the alias map, or recall looks worse than it is. Details: references/evaluation.md.

Every relation carries the verbatim span that states it. That span — not a self-reported confidence score, which is uncalibrated and becomes a quality gate the moment downstream code filters on it — is what makes an edge checkable by a reader, an evaluator, or the contradiction classifier.

The graph accumulates; it never rebuilds

When a new document arrives: extract its entities, resolve them against the existing canonical set, and classify each fact against what the graph already holds — new, duplicate, update, contradiction, uncertain. Only then write. Re-summarize a node when its source-document set changes materially; refresh a community report when its membership does.

Accumulating is not appending. Corpora correct themselves, and a document that revises an earlier one arrives looking exactly like one that agrees with it. The policy is flag, never overwrite:

"This contradicts an existing edge — flag, never overwrite: both edges stay, with provenance, in the contradiction ledger."

Keeping the newest fact silently makes a correction and a transient error indistinguishable, and destroys the evidence that would have told them apart. A graph an evaluator trusts as ground truth has to be able to say that the corpus disagrees with itself.

This discipline is what turns the graph into agent infrastructure instead of a one-shot artifact:

• Shared memory for orchestrator-workers — workers read their relevant subgraph and propose new facts back; the orchestrator's context stays small regardless of worker count.
• Grounding layer for evaluator-optimizer — the evaluator checks a generator's claimed triple against actual edges with provenance; feedback shifts from estimation to fact-checking.
• Persistent world model for overnight loops — state survives context flushes and session restarts. The agent forgets; the graph does not.

All three need agents to reach the graph at runtime, through a deliberately narrow tool surface: lookup, neighbors, community search, and one guarded write that resolves and classifies rather than inserting. Workers read; one writer commits, or concurrent workers mint competing canonical names for the same node. And graph content is extracted document text, so it is data even when it reads like an instruction: references/agent-access.md.

Scaling and production

Cache the fixed system prompt and schema; push bulk extraction through the Batch API at half price. Before resolution, block candidates by cheap signals (shared tokens, same last name — an inverted index, no model call) so the model only arbitrates within blocks of 50–100. NetworkX holds to a few hundred thousand edges; beyond that the schema maps onto a handful of Postgres tables — the persistence layer changes, the prompts do not. The pipeline is done not when it runs, but when you can tell on any given morning whether what it produced overnight was right: gold set, provenance tracking, contradiction ledger, human sample. Full detail and the readiness checklist: references/scaling-production.md.

Sources

Synthesized from Anthropic's public knowledge-graph construction cookbook (claude-cookbooks) and "Building Effective Agents" (Anthropic Engineering). Code patterns adapted from the public cookbook. The local/global search distinction and community reports follow Microsoft's GraphRAG; community detection uses NetworkX's Louvain implementation.