RAG Drift

RAG Drift

Drift occurs when the index contains chunks from files that have been deleted or modified since indexing. The retrieval system silently returns outdated or orphaned content without error. Detect and fix drift before it degrades quality.

What is drift?

Two types of drift corrupt the index:

Drift is silent. Retrieval still works, but returns wrong or outdated info.

Detect drift

Run the report script:

cd ~/.claude/rag-index
venv/bin/python report.py

Read the drift summary:

cat ~/.claude/rag-index/weekly.md | grep -A 30 "Stale chunks"

Example output:

Stale chunks by type:
- Missing (deleted): 3 chunks
  ~/.claude/old-skill/SKILL.md (deleted 2024-12-15)
  
- Modified (sha mismatch): 12 chunks
  ~/.claude/standards/old-auth.md (last indexed 2024-12-10, modified 2024-12-18)
  /Volumes/External\ HD/Desenvolvimento/forgekit/src/utils.ts (last indexed 2024-12-15, modified 2024-12-16)

Drift thresholds:

• <5 stale chunks: normal churn; can wait for next full rebuild
• 5–20 chunks: fix incrementally today

• 20 chunks: full rebuild recommended (faster than 20+ incremental ops)

Fix missing chunks (deleted files)

For files that no longer exist, delete their orphaned chunks:

sqlite3 ~/.claude/rag-index/index.sqlite

In the sqlite prompt:

-- View chunks from the deleted file
SELECT COUNT(*), path FROM chunks 
WHERE path LIKE '%old-skill/SKILL.md%'
GROUP BY path;

-- Delete them
DELETE FROM chunks 
WHERE path LIKE '%old-skill/SKILL.md%';

-- Confirm deletion
SELECT COUNT(*) FROM chunks 
WHERE path LIKE '%old-skill/SKILL.md%';

Exit sqlite: .exit

Quick delete (one line):

sqlite3 ~/.claude/rag-index/index.sqlite \
  "DELETE FROM chunks WHERE path = '~/.claude/old-skill/SKILL.md';"

Fix modified chunks (updated files)

Files that exist but have changed since indexing need incremental reindex:

cd ~/.claude/rag-index
venv/bin/python build.py --incremental ~/.claude/standards/old-auth.md

The incremental reindex will:

1. Delete old chunks from that file
2. Re-embed and insert new chunks with current content
3. Leave all other chunks untouched

Auto-drift detection

The sessionstart hook (sessionstart-drift-reindex.sh) runs at session start:

• Checks for >10 stale chunks
• Recommends full rebuild if drift is significant
• Logs to ~/.claude/rag-index/drift.log

This skill is for manual drift intervention. Use when:

• The auto-detector misses something
• You want granular control (delete vs. reindex vs. rebuild)
• Troubleshooting index corruption

Prevent drift

• Run report.py weekly to catch drift early
• After bulk file deletions, run a full rebuild
• After significant content updates, incremental reindex those files
• Archive old docs instead of deleting; move to ~/.claude/archive/ instead of rm

The .last-drift-reindex marker

After fixing drift, optionally update the marker file:

touch ~/.claude/rag-index/.last-drift-reindex

This marks the time drift was last resolved. The sessionstart hook checks this to avoid redundant corrections.

Troubleshooting

"Stale chunks count is high but report looks clean"

• Run report.py again to refresh: venv/bin/python report.py
• Check the report timestamp: stat ~/.claude/rag-index/weekly.md

"Incremental reindex didn't fix drift"

• Verify the file path is correct and the file exists
• Run report.py again to see if stale count changed
• If still high, use full rebuild

"Full rebuild but chunks didn't decrease"

• Check if new files were added at the same time
• Query count may be stable if source repos are unchanged
• Run inspect skill to audit what's in the index

See also

• adt-rag-index-rebuild — full or incremental reindex after drift
• adt-rag-quality — check retrieval quality; zero-hits may indicate drift
• adt-rag-inspect — examine chunks in the database