vault-query-bm25
Lexical retrieval only. Use when the answer's anchor words are likely present in the corpus verbatim — proper nouns, acronyms, exact phrases. For prosey, conceptual, or multi-hop questions, prefer vault-query-qkb (adds LLM query expansion, vector recall, graph expansion, and cross-encoder rerank).
Scope
- Vault root:
/Users/dmestas/projects/<vault>-kb/wiki
- qkb index: default (
~/.cache/qkb/index.sqlite) — the user's machine-wide qkb DB
- qkb collection:
<vault> (this vault's slice)
The default qkb DB hosts every collection on the machine. This skill always passes -c <vault> so results stay scoped to the vault. Drop or swap -c to reach other collections; index-wide ops (update, status) span every collection.
Default workflow
Rewrite the user question as compact keywords. BM25 has no query expansion — prosey questions get weak hits. Strip stopwords and verbs; keep proper nouns, acronyms, technical terms, and the 2-4 most discriminative content words.
User: "What are the guardrails around schema drift in our aviation data sources?"
Compact: "schema drift manifest fingerprint aviation data"
User: "How does FAA NMS handle NOTAM updates?"
Compact: "FAA NMS NOTAM updates" # proper nouns dominate; keep verbatim
Run BM25 search:
qkb search "$COMPACT_QUERY" --json -n 10 -c <vault>
Apply result hygiene:
- De-duplicate by resolved file path before reading.
- Prefer
wiki/entities/, wiki/concepts/, wiki/sources/, wiki/questions/, wiki/comparisons/ — synthesis-ready pages.
- Demote (don't discard)
hot.md, index.md, log.md, wiki/meta/*. Useful for orientation, not citation-grade evidence.
- If top hits are empty, all duplicates, or only meta/hub pages: retry with exact proper nouns/acronyms only, then with the primary keywords plus 2-4 domain terms. If still sparse, escalate to
vault-query-qkb.
Convert qkb URIs to file paths:
qkb://<vault>/wiki/entities/foo.md
-> /Users/dmestas/projects/<vault>-kb/wiki/entities/foo.md
Strip the qkb://<vault>/ prefix and prepend the vault root. Filenames may contain spaces — pass them literally to Read.
Read selected files with the Read tool. 3–5 is usually plenty; if BM25 is the right tool for the question, the top few hits will dominate.
Synthesize with wikilink citations, same shape as vault-query:
(Source: [[Page Title]])
When BM25 is the right tool
| Situation |
This skill |
vault-query-qkb |
| Answer anchors on proper nouns / acronyms / exact phrases |
✅ |
|
| Smoke check ("does the vault have anything about X?") |
✅ |
|
| Cheap AB-test baseline against the full pipeline |
✅ |
|
| Cost-sensitive: high query volume, tight latency budget |
✅ |
|
| Conceptual / prosey question, no obvious keyword anchor |
|
✅ |
| Multi-hop synthesis, comparison, gap analysis |
|
✅ |
| Vocabulary mismatch likely (user words ≠ vault words) |
|
✅ |
Cost: BM25 is ~50ms wallclock per query, ~50 tokens of output. The full qkb pipeline is 5–30s and an order of magnitude more output.
AB test protocol (vs vault-query / vault-query-qkb)
When the user asks for a comparison, report:
- Query text (verbatim).
- Compact keyword query actually used.
- qkb command used.
- Top hits: title, path, score.
- Files actually read.
- Wall-clock time end-to-end.
- Whether BM25 alone was sufficient or escalation was needed.
Don't tune queries mid-test. Don't edit the vault mid-test.
Index maintenance
qkb update # re-index changed files (spans all collections)
qkb status # doc + embedding status
BM25 doesn't need embeddings — qkb search runs purely on the FTS5 index, which update keeps fresh. Do NOT run qkb embed from this skill; that's the vector pipeline's concern.
Guardrails
- Treat BM25 output as retrieval hints, not citations by itself. Always
Read the actual wiki files before making source-backed claims.
- Do not use this skill to ingest, rewrite, or save wiki pages. That's
vault-save / vault-ingest territory.
- Lexical retrieval has known blind spots — paraphrase, synonyms, conceptual indirection. When user vocabulary doesn't match the vault's, escalate to
vault-query-qkb. Don't fabricate to fill the gap.
- This skill complements the full pipeline; it does not replace it. Keep
vault-query-qkb as the default for non-trivial questions.
1---2name: vault-query-bm253description: Fast BM25 (lexical-only) retrieval over the Flight Planner wiki via qkb. No LLM, no graph, no rerank — cheap and deterministic. Use for known-term questions (proper nouns, acronyms, exact phrases), AB-test baselines against the full pipeline, and quick smoke checks. Triggers on: vault-query-bm25, bm25 vault query, lexical vault search, fast keyword search the wiki, qkb bm25, search the wiki without LLM.4---56# vault-query-bm2578Lexical retrieval only. Use when the answer's anchor words are likely present in the corpus verbatim — proper nouns, acronyms, exact phrases. For prosey, conceptual, or multi-hop questions, prefer `vault-query-qkb` (adds LLM query expansion, vector recall, graph expansion, and cross-encoder rerank).910## Scope1112- Vault root: `/Users/dmestas/projects/<vault>-kb/wiki`13- qkb index: default (`~/.cache/qkb/index.sqlite`) — the user's machine-wide qkb DB14- qkb collection: `<vault>` (this vault's slice)1516The default qkb DB hosts every collection on the machine. This skill always passes `-c <vault>` so results stay scoped to the vault. Drop or swap `-c` to reach other collections; index-wide ops (`update`, `status`) span every collection.1718## Default workflow19201. **Rewrite the user question as compact keywords.** BM25 has no query expansion — prosey questions get weak hits. Strip stopwords and verbs; keep proper nouns, acronyms, technical terms, and the 2-4 most discriminative content words.2122 ```text23 User: "What are the guardrails around schema drift in our aviation data sources?"24 Compact: "schema drift manifest fingerprint aviation data"2526 User: "How does FAA NMS handle NOTAM updates?"27 Compact: "FAA NMS NOTAM updates" # proper nouns dominate; keep verbatim28 ```29302. **Run BM25 search:**3132 ```bash33 qkb search "$COMPACT_QUERY" --json -n 10 -c <vault>34 ```35363. **Apply result hygiene:**37 - De-duplicate by resolved file path before reading.38 - Prefer `wiki/entities/`, `wiki/concepts/`, `wiki/sources/`, `wiki/questions/`, `wiki/comparisons/` — synthesis-ready pages.39 - Demote (don't discard) `hot.md`, `index.md`, `log.md`, `wiki/meta/*`. Useful for orientation, not citation-grade evidence.40 - If top hits are empty, all duplicates, or only meta/hub pages: retry with exact proper nouns/acronyms only, then with the primary keywords plus 2-4 domain terms. If still sparse, escalate to `vault-query-qkb`.41424. **Convert qkb URIs to file paths:**4344 ```text45 qkb://<vault>/wiki/entities/foo.md46 -> /Users/dmestas/projects/<vault>-kb/wiki/entities/foo.md47 ```4849 Strip the `qkb://<vault>/` prefix and prepend the vault root. Filenames may contain spaces — pass them literally to `Read`.50515. **Read selected files** with the `Read` tool. 3–5 is usually plenty; if BM25 is the right tool for the question, the top few hits will dominate.52536. **Synthesize with wikilink citations**, same shape as `vault-query`:5455 ```text56 (Source: [[Page Title]])57 ```5859## When BM25 is the right tool6061| Situation | This skill | vault-query-qkb |62|-----------|:---:|:---:|63| Answer anchors on proper nouns / acronyms / exact phrases | ✅ | |64| Smoke check ("does the vault have anything about X?") | ✅ | |65| Cheap AB-test baseline against the full pipeline | ✅ | |66| Cost-sensitive: high query volume, tight latency budget | ✅ | |67| Conceptual / prosey question, no obvious keyword anchor | | ✅ |68| Multi-hop synthesis, comparison, gap analysis | | ✅ |69| Vocabulary mismatch likely (user words ≠ vault words) | | ✅ |7071Cost: BM25 is ~50ms wallclock per query, ~50 tokens of output. The full qkb pipeline is 5–30s and an order of magnitude more output.7273## AB test protocol (vs `vault-query` / `vault-query-qkb`)7475When the user asks for a comparison, report:7677- Query text (verbatim).78- Compact keyword query actually used.79- qkb command used.80- Top hits: title, path, score.81- Files actually read.82- Wall-clock time end-to-end.83- Whether BM25 alone was sufficient or escalation was needed.8485Don't tune queries mid-test. Don't edit the vault mid-test.8687## Index maintenance8889```bash90qkb update # re-index changed files (spans all collections)91qkb status # doc + embedding status92```9394BM25 doesn't need embeddings — `qkb search` runs purely on the FTS5 index, which `update` keeps fresh. Do NOT run `qkb embed` from this skill; that's the vector pipeline's concern.9596## Guardrails9798- Treat BM25 output as retrieval hints, not citations by itself. Always `Read` the actual wiki files before making source-backed claims.99- Do not use this skill to ingest, rewrite, or save wiki pages. That's `vault-save` / `vault-ingest` territory.100- Lexical retrieval has known blind spots — paraphrase, synonyms, conceptual indirection. When user vocabulary doesn't match the vault's, escalate to `vault-query-qkb`. Don't fabricate to fill the gap.101- This skill complements the full pipeline; it does not replace it. Keep `vault-query-qkb` as the default for non-trivial questions.
danmestas