charlieviettq RAG systems
Pipeline stages
- Ingest — parse, clean, preserve structure (headings, tables).
- Chunk — size/overlap tuned to content type; keep metadata (source, section).
- Embed — consistent model; version stored with index.
- Retrieve — hybrid (keyword + vector) when recall matters.
- Rerank — optional cross-encoder for top-k precision.
- Generate — cite sources; refuse when context insufficient.
Chunking heuristics
| Content | Guidance |
|---|---|
| Docs | Split on headings; 300-800 tokens typical |
| Code | Function/class level; include path in metadata |
| Tables | Row batches or markdown table blocks |
Quality checks
- Retrieval hit rate on golden questions
- Answer grounded in retrieved chunks (no hallucinated citations)
- Latency budget per query documented
- Index refresh process defined
Failure modes
| Issue | Mitigation |
|---|---|
| Missed relevant doc | Hybrid search, query expansion, metadata filters |
| Wrong chunk | Smaller chunks, reranker, parent-child retrieval |
| Stale index | Version tag, scheduled re-embed |
| Prompt overflow | Summarize chunks, dynamic top-k |
Evaluation
- Golden Q&A set with expected source doc/section.
- Metrics: recall@k, answer correctness, citation accuracy.
- Regression when changing embed model or chunk strategy.
Security
- Respect ACLs at retrieval time; do not leak cross-tenant data in shared indexes.