# Tensor Grep Index Fingerprint Freshness > Use when working on index/reuse staleness detection — the Rust `.tg_index` reuse path, a cached parse/symbol/semantic index that is served on a LATER query, `staleness_reason` / `compute_tree_fingerprint` / mtime-size identity, or a "reused index must not serve the wrong tree" guard (M17). Covers identity-anchor-first (canonical root, not a raw path string), the alias/swap RED shape that a naive wrong-root comparison cannot reach, the fingerprint ladder (mtime+size → ctime → content-checksum → whole-tree), and bounded sampling with NAMED boundaries so the honest cap is a reopen trigger, not a silent hole. Triggers: "index reuse", "staleness", "fingerprint", "wrong tree", "stored root", "canonical_root_of", "mtime size", "ctime", "sampling cap", "M17", "swap". Sibling of tensor-grep-cross-platform-path-confinement (the alias swap is a junction there) and tensor-grep-hermetic-hostile-tests (the fixture must BITE); this one is the identity/freshness discipline for CACHED INDEX REUSE. - Skill: `oimiragieo/tensor-grep-index-fingerprint-freshness` (Agent Skill) - Install (CLI): `npx skillmds@latest add oimiragieo/tensor-grep-index-fingerprint-freshness` - Raw SKILL.md: https://api.skillmd.com/api/skills/oimiragieo/tensor-grep-index-fingerprint-freshness/raw - Safety review: pending - Works with: Claude Code, Claude.ai, OpenAI Codex - Category: Coding & Dev Tools - Author: oimiragieo (https://skillmd.com/u/oimiragieo) - Updated: 2026-09-17 - Page: https://skillmd.com/skills/oimiragieo/tensor-grep-index-fingerprint-freshness --- # tensor-grep: index fingerprint + reuse freshness An index is a memory of a PAST tree. Serving it on a LATER query is only correct if the past tree is still the queried tree — and "still the queried tree" is a WEAKER claim than "nothing changed", because the same path string can name a DIFFERENT tree (a swapped symlink/junction, a renamed tree, a copied/tampered `.tg_index`). The M17 finding (index reuse) was the exact case: `staleness_reason` checks only no-ignore mode + per-entry mtime/size + a new-file walk **over `self.root`** — it never asks "is the stored root the queried root, canonically?". --- ## Part 1 — Identity-anchor-first: canonical root, never a raw path string The M17 structural fix (merged on origin/main as v1.110.12; current tag v1.110.16 (re-derive with: git describe --tags origin/main)) is the model: - [ ] Persist a **canonical** root at build: `canonical_root_of()` recorded once (format v6) — lexical AND canonical, so later path-spelling differences (case, `..`, junctions in the prefix) cannot manufacture a mismatch that is not real. - [ ] Store entries **canonical-relative** to that root with a single deref (`canonical_root.join(rel)`), so NOTHING is cwd-dependent — a later query from a different working directory must not change where entries resolve. - [ ] At reuse/warm-load, compare **canonical QUERY root vs canonical STORED root** at EVERY load site — the `preloaded_index` arm handed off by `detect_warm_index_state` AND the fresh-reuse branch — and the check must run BEFORE `staleness_reason`/incremental update. - [ ] On mismatch: REBUILD from the current tree (always safe in-tree — prefer rebuild over bare refuse) with a disclosed reason — rebuild must not be invisible latency. Root bytes are already persisted (`index.rs`, `canonical_root` field — grep the symbol). Two corrections to an earlier pass (2026-08-12): the field is `canonical_root`, NOT `root` (the pre-fix plan this skill was drafted from used a bare `root` field; format v6 persists the canonical form, declared `canonical_root: PathBuf`); and the plan's line refs copied into that pass (`:272-274` write / `:355-356,413` read) were the PLAN's own pre-fix numbers, unlabeled — on this tree the write is the format-v6 serializer emitting `canonical_root_bytes` and the read is the deserializer parsing it back (re-grep `canonical_root` in `rust_core/src/index.rs` to relocate both). The defect was that the reuse path never COMPARED them to the query root. --- ## Part 2 — The alias/swap RED shape (the repro that actually bites) CRITICAL constraint: `resolve_index_path` puts `.tg_index` INSIDE the tree, so a plain "build root A, query root B" NEVER shares an index file — the naive RED is unreachable. The real wrong-tree serve needs an **ALIAS**: same string, swapped tree: - [ ] RED fixture = a root symlink/junction under a FIXED path, swapped to a different directory BETWEEN build and query (Windows: a junction — see tensor-grep-cross-platform-path-confinement Part 1); on a metadata-preserving swap this is the ONLY shape that still looks fresh. - [ ] The fixture carries the fixture-BITES precheck (prove the swap actually applied — the junction must resolve to the NEW dir before the probe runs; see tensor-grep-hermetic-hostile-tests). - [ ] A naive `stored_root == query_root` string compare PASSES on the alias (same string) — the canonical-root comparison is what separates "same text" from "same tree". - [ ] Keep the per-file identity walk (`staleness_reason`) as the "still the queried tree" backstop — it is not replaced, it is supplemented. --- ## Part 3 — The fingerprint ladder: climb only as far as the threat demands | Level | Signal | Evadable by | Use when | |---|---|---|---| | 1 | **mtime + size** (per entry) | a same-size/same-mtime edit (`touch -r`, an rsync that preserves mtime+size) | cheap per-entry freshness in `staleness_reason`'s loop — NEVER a tree-level claim | | 2 | **ctime** (change/inode time) | clock-restored file systems, a swap that preserves ctime — but catches `touch -r` on most real systems | an intermediate tier when same-mtime edits are in the threat model | | 3 | **content checksum** (per sampled file, FULL bytes) | almost nothing short of a genuine same-checksum collision | the sampled-file tier of `compute_tree_fingerprint` — the 4 KiB byte cap that let an edit past offset 4096 evade was a real finding | | 4 | **whole-tree** (every file fully hashed) | nothing short of a full collision | the M17-FU1 follow-up for unsampled files (below) | Receipts that shaped the ladder: the M17 gate round-2 finding that the initial 4 KiB-per-file byte cap let a same-size/same-mtime edit past byte 4096 in a SAMPLED file evade detection, and that inode identity would not catch it either — so sampled files are hashed in FULL (bounded by the 32-file cap, not by bytes). See `compute_tree_fingerprint` (`index.rs`, grep the symbol). --- ## Part 4 — Bounded sampling with NAMED boundaries (the reopen-trigger discipline, A49) `compute_tree_fingerprint` samples the **top-32** top-level FILES (direct children) of the canonical root — the walk filters to regular files BEFORE the cap (`entry.file_type()...is_file()` precedes `.take(TREE_FINGERPRINT_TOP_LEVEL_CAP)`), so a directory or symlink can neither flip the digest nor consume a sample slot — fully hashed, index-machinery namespace excluded (a leftover `..tg_index..tmp` atomic-save temp / `..tg_index.lock` must not consume a sample slot — `is_tg_index_owned_entry`, whose matcher is `starts_with("..tg_index.")` — TWO leading dots, not one). The honest remaining boundary is NAMED, not hidden: - [ ] The cap (`TREE_FINGERPRINT_TOP_LEVEL_CAP = 32`) is documented AT the constant with the reason ("deliberately bounded in FILE COUNT; each sampled file is FULLY content-hashed"). - [ ] The files NOT sampled (33rd+ top-level files, and every NON-top-level file) are covered only by the per-file mtime/size loop — that gap is **M17-FU1**, with owner + reopen trigger, recorded canonically (see docs/plans/2026-08-08-backlog-completion-plan.md M17 section and BACKLOG's M-follow-up rows). - [ ] A sampling cap is an honest tradeoff, but it must be a *named* tradeoff with a reopening trigger (A49), never a silent assumption — "a same-path metadata-preserving swap landing in an unsampled file that mtime/size cannot detect" is that trigger. The M17 REOPEN TRIGGER pattern applies to every fingerprint surface in the repo, including the Python parse/semantic caches: - `semantic_index.py` stale check = SHA-256 fingerprint over sorted paths + mtimes; on mismatch it warns to stderr and falls back in-memory (NOT wired to a `tg index` command yet). - `repo_map.py` `_mtime_aware_cache` (grep the symbol) is the Python cache-freshness wrapper with a `_MTIME_CACHE_CLEAR_REGISTRY` that sweeps every decorated cache — and the per-root keyed re-export/tsconfig contexts ARE swept too, contrary to an earlier pass of this skill that said they were "deliberately NOT" among them: `_clear_all_source_caches` (grep the symbol in `repo_map.py`) explicitly clears `_JS_TS_REPO_CONTEXTS`, `_RUST_REPO_CONTEXTS`, and calls `lang_go.clear_go_repo_context_cache()` with the comment "Clear them too so the sweep is actually complete" — they are not `_mtime_aware_cache` wrappers, so the sweep adds them by hand. --- ## External anchors (Exa research, 2026-08-09) | Anchor | Their point | This skill's mapping | |---|---|---| | **rsync/backup-tool mtime-size semantics** (rsync(1) manpage; rsync issue/fix #627 for ctime handling in some tools) | mtime+size is cheap but is the classic evadable signature — restoring mtime+size is a standard attack on freshness heuristics | Part 3's ladder: why you climb to ctime/checksum when same-mtime edits are in the threat model. | | **"Do not trust mtime alone for cache invalidation"** (general build-tool / ccache / ninja literature; the same reason `touch -r` defeats incremental builds) | File-system metadata timestamps are attacker-influenceable (or accidentally restorable) | Level 1's "NEVER a tree-level claim" + the 4 KiB-cap receipt. | | **Content-addressed / digest-bounded verification (general)** | A bounded digest is a decision about the threat's cost, not a correctness shortcut | Part 4's named sampling cap + reopen trigger discipline. | **Repo receipts to cite by symbol, not line:** `index.rs` `root` / `staleness_reason` / `compute_tree_fingerprint` / `TREE_FINGERPRINT_TOP_LEVEL_CAP` / `is_tg_index_owned_entry`; `main.rs` `resolve_index_path` / the reuse branch / `detect_warm_index_state`; `src/tensor_grep/core/semantic_index.py` (the SHA-256 mtime-path fingerprint + fallback); `repo_map.py` `_mtime_aware_cache` / `_MTIME_CACHE_CLEAR_REGISTRY`; AGENTS.md A44/A49 and the verification-oracle family (a mismatch that is not real is a Form-10 branch-unit false green). Grep the symbol, never a hardcoded line number; and record every deferred fingerprint boundary as a named M-FU row with owner + reopen trigger. --- ## Quick reference ``` [1] identity canonical query root vs canonical STORED root, at BOTH load sites, BEFORE staleness [2] REPRO wrong-tree serve needs an ALIAS/SWAP (junction), never the naive wrong-path RED [3] ladder mtime+size -> ctime -> full-content checksum -> whole-tree; climb to the threat [4] named cap 32-file sample is bounded AND the unsampled gap is M17-FU1 with an owner+trigger [5] mismatch rebuild with a disclosed reason, never invisible latency, never bare refuse ``` The endpoint: an index that never serves a tree it was not built for, whose freshness proof is as strong as the threat requires, and whose sampling cap is an honest, named, reopenable boundary.