Corpus Retrieval Lab

Benchmark current research-query and direct lexical baselines against experimental vector, BM25, typed concept-graph PPR, and RRF source selection without replacing production retrieval.

jmagly Updated

File contents

Corpus Retrieval Lab

Run the local experimental benchmark:

aiwg corpus retrieval-lab --queries <jsonl> --concepts <json> --json

Use reviewed narrow implementation questions with expected REF evidence. Preserve the current research-query behavior regardless of the result; the report may only recommend a later explicit adoption decision.

The lab reports baseline and hybrid Hit@k, MRR, latency, failures, confidence/dispersion, faithfulness, and concept-scheme drift. Use --expected-scheme-hash when comparing runs across time.

See docs/guides/corpus-retrieval-lab.md for fixture schemas and interpretation.

jmagly/ai-writing-guide/tree/main/agentic/code/frameworks/research-complete/skills/corpus-retrieval-lab commit 23f259bad2

Frequently asked questions

npx skillmds@latest add jmagly-ai-writing-guide/corpus-retrieval-lab