Xerj Code

XERJ.code — reference coding

An agent that does not know an API guesses, runs, fails, and guesses again. Each lap costs output tokens, which are the expensive kind. This skill replaces laps with a lookup: clone the repositories that already contain a correct implementation, index them with XERJ, and retrieve the passage before writing.

The trade is deliberate. Retrieval spends input tokens, which are cheaper per token and cacheable; loops spend output tokens. See COSTS.md for the arithmetic and for the conditions under which this trade loses.

Status, stated plainly: measured, and the value is real but narrow. (Full case study with real tokens and dollars: docs/case-studies/reference-coding/CASE_STUDY.md, 2026-08-05.)

The comparison that matters is against native Claude Code — the same agent with tools that would grep the source itself — not a bare model. Measured across three regimes, same agent, one objective verdict (compiles + passes hidden tests):

• Unfamiliar code with a contract the model can't recall (a seal, a generational handle, a lazy refill, a specific hash scheme — measured across seven purpose-built libraries, 21 runs): native and xerj both solve 21/21; xerj uses 1.5× fewer output tokens and costs 1.3× less than native, because retrieval replaces the grep. Pure memory fails 1/21 and burns 6.5× the cost flailing. This is the win, and it is decisive. The compiler can leak an API name across a retry loop, but never a contract.
• Memorised code (any popular public crate — even a 256-value table, or a protobuf-style varint): retrieval is overhead. Pure memory is cheapest; injecting a big reference can be the worst arm, because the model re-emits what it was handed.

So: use this when the reference is code the model has not memorised — your own private/proprietary code, an internal API, a post-cutoff or niche library. On public library references the model already knows, it costs more than it saves. The native-vs-xerj gap grows with corpus size (grep pulls the whole tree into context; retrieval pulls one passage).

Standing corpora for this repository (MANDATORY since 2026-08-06)

In the xerj repo, retrieval-before-writing is required for non-trivial engine work, not optional. Each corpus below has a pinned, licence-reviewed definition in hub/ — xc-corpus.sh --from hub/<name>.json rebuilds it at the same commits anyone else is using:

Approach-only sources: elasticsearch (AGPL-3.0 / SSPL-1.0 / Elastic-2.0) and sonic (MPL-2.0). Never copy their code into Apache-2.0 XERJ; for ES, pasting source would also falsify the project's public "shares no code with Elasticsearch" claim. meilisearch is MIT at the core but BUSL-1.1 for its Enterprise Edition parts — check the header/path first. Read the design, write our own implementation. Each manifest's review.use field says which bucket a repo is in; hub/README.md explains the three.

Operational traps, both hit in practice

• Nothing under /tmp. Clones live in ~/.xerj-code/corpora/, the index in ~/.xerj-code/data. A /tmp data dir is lost on reboot and the whole corpus silently retrieves nothing.
• --fresh is required after the data dir is wiped or moved. autoindex keeps incremental state in ~/.xerj/autoindex/; run it without --fresh against an empty instance and every file is skipped as "already indexed", leaving 0 documents and a corpus that returns nothing without erroring. Always verify: curl -s "$URL/xc-<corpus>*/_count" must be > 0.

When this is worth it

Use it when the task touches code someone else has already written correctly:

• an unfamiliar library, framework, or protocol
• porting an algorithm whose reference implementation exists
• matching an established convention across a large codebase
• any time you have already looped twice on the same error — that is the signal that guessing is not converging

Do not use it for code that only exists in this repository (ordinary file reads are cheaper), for one-line edits, or when you already know the API. Indexing has a fixed setup cost; a task you would finish in two tool calls will not repay it.

The loop

xc-corpus.sh <name> <git-url>...   # clone reference repos (once per domain)
xc-corpus.sh --from <manifest>     # ...or rebuild a corpus someone else defined
xc-index.sh  <name>                # index them with xerj autoindex (once per corpus)
xc.py <name> "<what you need>"     # retrieve before writing (every task)

1. Build a corpus

Group repositories by problem domain, not by language. A corpus for "async Rust" holds tokio, hyper, and tower; a corpus for "parsers" holds nom, pest, and tree-sitter. Keep corpora small and sharp — a corpus that contains everything retrieves like a search engine with no query.

scripts/xc-corpus.sh async-rust \
  https://github.com/tokio-rs/tokio \
  https://github.com/hyperium/hyper

Clones are shallow (--depth 1) and land in ~/.xerj-code/corpora/<name>/. Every build writes a corpus.json beside them — URLs, full commit SHAs and licences, a few hundred bytes, no source. That file is the shareable corpus definition: hand it to someone else and

scripts/xc-corpus.sh --from their-corpus.json

rebuilds the same commits on their machine (an existing clone is moved to the recorded SHA, so both trees match). Vetted definitions for the corpora this project uses live in hub/:

scripts/xc-corpus.sh --from hub/xerj-storage.json && scripts/xc-index.sh xerj-storage

2. Index it

scripts/xc-index.sh async-rust

This runs xerj autoindex against the local XERJ instance. Exit code 3 means "completed with junk files" and is not a failure — some files in any real repository are unparseable. Treat 0 and 3 as success; anything else is real.

3. Retrieve before writing

scripts/xc.py async-rust "graceful shutdown with a broadcast channel"

xc.py returns the matching definition, not a byte window:

─── valkey/src/networking.c  (score 12.68, BSD)
    [function addBulkStringToReplyIOV @ line 2635 — 366 of 272,498 chars]
static void addBulkStringToReplyIOV(char *buf, size_t buf_len, ...) {

That line number is the whole point. A record is one whole file, so ranking can only tell you which file matched; the definition is located from the symbols[] the index already carries. Before this, an 8 KB slice of a 272 KB file was a coin flip on whether the answer was inside it — a query about null replies correctly ranked networking.c and then returned its licence banner, because addReplyNull lives at line 1460. Read the named definition, cite it, and if the top hits are irrelevant say so and fall back to normal work rather than forcing them in.

--full N caps each passage at N chars. --no-symbol falls back to a raw window (only useful for data files with no symbols at all).

Retrieval modes

The default is bm25. Measured across two corpora and twelve hand-labelled queries:

top-3 is the operative metric — the agent reads k passages, not one — and BM25 is perfect on it while never being the worst arm on either corpus. It is also ~5× faster, with no vector round trip and no mapping lookup.

Hybrid was previously the default, chosen on rust-text alone where it wins top-1. That was a corpus-specific result: on the C corpus the vector arm reaches only 5 of 407 indices (issue #173), so fusion mixes a good ranking with one that cannot see 98.8% of the material. Use --mode hybrid when a corpus has broad semantic_text coverage and you specifically care about top-1 — and measure it before trusting it.

scripts/xc.py rust-text "lazy DFA cache eviction"                 # bm25 (default)
scripts/xc.py rust-text "lazy DFA cache eviction" --mode hybrid   # + vector, RRF-fused
scripts/xc.py rust-text "lazy DFA cache eviction" --mode semantic # vector only

A hybrid answer opens with a line naming the arms that actually ran. The vector arm only works where body is mapped as semantic_text, and a semantic query against an index where it is not does not degrade — it fails the whole search with a 400 and takes every other index in the wildcard with it. xc.py reads the mapping first and aims the vector arm only at capable indices, so a corpus with no semantic_text says BM25 only — ... rather than pretending.

xc.py never requests highlight. On this engine a highlight block changes _score and reorders hits (issue #177): identical query, top-1 6/6 without it and 1/6 with it. Since passages now come from symbols[], the highlighter bought nothing and cost ranking.

Rules that keep this honest

Retrieved code is evidence, not authority. A passage proves how that project solved the problem under its constraints. Cite file:line when you rely on it, and say plainly when you are adapting rather than copying.

Check the licence before copying. xc-corpus.sh records each repository's licence in corpus.json. Copying a GPL implementation into a permissively licensed project is a real problem, not a technicality. Adapt the approach and write your own code when the licences are incompatible.

A stale index is worse than none. It returns code that no longer exists and sends the agent down a dead path with false confidence. xc.py refuses to answer from an index older than 30 days unless you pass --stale-ok; re-run xc-index.sh instead.

Never index the working repository into a shared corpus. Reference corpora are for other people's code. Mixing your own in makes retrieval return your own mistakes as precedent.

MEATL

For agent-to-agent and agent-to-tool exchanges, use the notation in MEATL.md — a compact, checkable format that cuts output tokens on the messages no human reads. It is explicitly not for user-facing text: prose to a person stays prose. A summary compressed into arrow chains costs the reader more than it saves the budget.

Measuring it

The published numbers, the per-run raw data and the price model live in docs/case-studies/reference-coding/: CASE_STUDY.md for the write-up, COSTS.md for why output tokens are the expensive kind, data/results-*.json for the runs behind each figure, and generated/ for code the two arms actually produced.

The measurement harness itself (csrun.py, the purpose-built reference libraries, the hidden-test task specs) is not shipped in this repository — only the retrieval tooling in this directory is. What that harness does, if you want to build the equivalent: run three arms of the same agent — bare (memory only), native (it greps the corpus tree itself), xerj (the retrieved passage is injected) — over tasks whose verdict is objective, i.e. the generated code compiles and passes a hidden test the model never sees. Report medians, not means: one flailing trial can be 5× another.

Two rules that decide whether a result means anything:

• Validate the tasks before spending a token. Inject the reference-correct solution (must pass) and a plausible wrong one — correct but quadratic, or missing the contract — which must fail. A benchmark whose tests pass wrong code measures nothing.
• Do not measure on code the model has memorised. It reproduces even a 256-value table from recall, and retrieval then shows as pure overhead. The honest test bed is code that was never in a training set — your own private code is the realistic case.

After the session: file the field report

Retrieval is only half the loop. The other half is telling the maintainers what using XERJ was actually like — the baseline contribution asked of every agent that uses it (user-feedback/16-agent-field-reports/README.md, https://xerj.org/llms.txt). One command drafts it and fills in the facts (version, OS, what was indexed) for you:

# opens the PR (branch + commit of ONLY the report + gh pr create):
xerj feedback --open-pr --used-for "reference coding" \
  --verdict "…what worked, what did not…"

# sandboxed and cannot push? draft it and print the exact git+gh commands:
xerj feedback --dry-run --used-for "reference coding" --verdict "…"

It only ever auto-fills facts it can observe and leaves every opinion as a placeholder for you to fill — never an invented verdict. A pull request that adds only a field report is exempt from the CLA gate, so this costs a signature nothing. xerj feedback --help has the full contract.