Tensor Grep Config And Flags

tensor-grep config and flags

A ground-truthed catalog of every tg config axis — env vars, CLI flags, provider modes — plus the registration checklist for adding a new one. Verified against source as of 2026-07-23, v1.95.0 (pyproject.toml). Re-verify commands are in Provenance and maintenance because these drift with every release.

When to use this skill

• You are adding, renaming, or removing an env var or CLI flag.
• A search flag is reaching ripgrep raw (rg: unrecognized flag at runtime) or a command 404s.
• You need to know whether a knob (--gpu-device-ids, --provider lsp, TENSOR_GREP_CLASSIFY_PROVIDER=cybert) is production-safe to recommend to a user, or still experimental/default-off.
• You need the authoritative default value or guard condition for a TG_* / TENSOR_GREP_* variable before writing docs, a benchmark harness, or an agent prompt that references it.

When NOT to use this skill (use the sibling instead)

The two front doors, and why config is split across them

tg has two CLI entry points that both parse flags, and a config change that only lands in one is a silent bug, not a crash:

1. Python bootstrap (src/tensor_grep/cli/bootstrap.py) — tensor_grep.cli.bootstrap:main_entry intercepts plain-text searches before the Typer app loads and forwards them straight to rg.
2. Rust native front door (rust_core/src/main.rs) — the standalone tg binary, which re-implements flag parsing with clap.

The canonical, always-current documentation of every env var and the full search flag surface lives in two places that are meant to stay in sync — read these first when you need ground truth fast:

• tg --help epilog: src/tensor_grep/cli/main.py:187-200 (the app = typer.Typer(help="""...""") block).
• Native tg --help epilog: ENVIRONMENT_OVERRIDES_HELP const, rust_core/src/main.rs:67.

If those two drift from each other or from this file, trust the source, not this document — see Provenance and maintenance.

Environment variable catalog

Boolean env vars in tg follow one convention almost everywhere (env_flag_enabled, grep -n "def env_flag_enabled" src/tensor_grep/cli/runtime_paths.py — was :13-15, now :20-22): the raw value is lower-cased and stripped, and it is "on" only if it is exactly 1, true, yes, or on — anything else (including unset) is "off". Two STRICT exceptions compare the raw value against the literal "1" only, so true/yes/on do NOT enable them: TG_DOCTOR_OFFLINE is == "1" on the Python side (grep -n "TG_DOCTOR_OFFLINE" src/tensor_grep/cli/main.py — :492), and TG_RESIDENT_AST is != "1" on the Rust side (grep -n "TG_RESIDENT_AST" rust_core/src/main.rs — :7581).

Routing / launcher

Timeouts

Every non-positive or unparseable value for the float-typed timeout vars above silently falls back to the compiled-in default (_configured_positive_float, subprocess_policy.py:9-17) — a bad value does not crash, it just gets ignored. Do not assume "I set it, therefore it changed" without checking tg doctor --json's env block (see Discovering effective config).

GPU

Fail-loud contract: an explicit --gpu-device-ids request that cannot be honored raises ConfigurationError (a RuntimeError subclass, pipeline.py:20-21) — it never silently falls back to CPU. Example message shape (pipeline.py:26-39):

GPU acceleration is experimental. Explicit GPU device selection [0, 1] could not initialize a
GPU backend: fixed-string (-F) search has no GPU backend

This is deliberate: -F/--fixed-strings GPU search has no kernel yet, so pairing it with an explicit --gpu-device-ids must error, not silently drop to CPU and report a clean result (see the Backend Fail-Closed Contract in AGENTS.md:214-224, and tensor-grep-architecture-contract for the general principle). By contrast, the heuristic (non-explicit) auto-GPU path degrades to CPU with a visible warnings.warn(...) + fallback_reason, not an exception — only explicit user intent gets fail-loud treatment (pipeline.py:174-176, _should_honor_explicit_gpu_ids).

GPU remains EXPERIMENTAL end-to-end. GPU Phase-0 SHIPPED (v1.75.0-v1.75.4, PRs #593-#597): NVIDIA native assets are built and locally correctness-proven (RTX 4070 sm_89 / RTX 5070 sm_120 -- docs/gpu_crossover.md), but gated OFF the public release by the CI Actions var TENSOR_GREP_RELEASE_NATIVE_ASSET_PROFILE (default native-frontdoor, CPU-only; GPU asset publishing needs the non-default native-frontdoor-gpu) -- Phase 1 is now a reversible flag-flip, not a multi-week rebuild. That flip publishes assets only: no speed crossover is proven vs rg/tg_cpu, GPU auto-recommendation stays false, and the reviewer-gated public-gpu-proof.yml speed-crossover gate remains unmet (docs/CONTRACTS.md:80-82). Do not market or default-enable it.

Classify provider

tg classify always reports provenance in its JSON output (classification_backend / provider_requested / provider_used / provider_status / fallback_reason / cache, sidecar.py:57-71) so a caller can distinguish "asked for cybert, got heuristic because it failed" from "asked for heuristic". Never read a classify result as model-backed without checking provider_used.

Session / daemon

The daemon binds to 127.0.0.1 only (session_daemon.py:46) — it is not exposed off-host. Operational detail (starting/stopping the daemon, tg session daemon start|status|stop) lives in .claude/skills/tensor-grep/REFERENCE.md, not here.

Agent capsule

MCP security gate (default-OFF)

This is an Enablement Discipline case: default-OFF, opt-in only, and it is the kind of knob you should never flip on in a shared/CI MCP server config without an explicit operator decision — see AGENTS.md "Enablement Discipline (autonomous behaviors)" (referenced from the workspace root CLAUDE.md) and tensor-grep-change-control for the graduation gate (council-verify → dry-run → conscious flag-flip).

Evidence signing (tg evidence emit --sign / tg evidence verify)

Ledger (tg ledger claims / findings)

In-process caches (bound long-lived agent-loop state)

These exist so a long-lived tg session daemon / MCP server process doesn't grow unbounded caches. All are documented together in main.py:199-200; defaults are implementation-internal (read the cited module if you need the exact number) — this skill's job is to tell you that they exist and where, not to duplicate the numeric defaults, which drift independently of flags/commands.

• TENSOR_GREP_CPU_LITERAL_INDEX_CACHE_MAX_ENTRIES
• TENSOR_GREP_STRING_INDEX_CACHE_MAX_ENTRIES
• TENSOR_GREP_AST_QUERY_CACHE_MAX_ENTRIES
• TENSOR_GREP_AST_NODE_INDEX_CACHE_MAX_ENTRIES
• TENSOR_GREP_REPO_CONTEXT_CACHE_MAX_ROOTS
• TENSOR_GREP_LSP_PROVIDER_CLIENT_CACHE_MAX_ENTRIES
• TENSOR_GREP_LSP_PROVIDER_OPEN_DOCUMENT_MAX_ENTRIES

Internal constants (not env-configurable)

Not every load-bearing bound in tg is an environment variable — some are deliberately hardcoded constants, single-sourced so multiple call sites cannot drift apart. Distinguish these from the env-configurable knobs above before assuming a behavior can be tuned at runtime:

• IMPLICIT_SEARCH_WALK_FILE_CEILING = 1500 (DEFINED at src/tensor_grep/io/scan_limits.py:106; io/directory_scanner.py only RE-EXPORTS it. An earlier revision cited directory_scanner, which would send someone changing the ceiling to edit a re-export and wonder why nothing moved — matches the sibling tensor-grep-architecture-contract A9 wording) — the fast-refuse ceiling for an unscoped/defaulted-path search or tg find walk (A9, v1.92.3/#702). It is imported by both src/tensor_grep/cli/main.py's _LARGE_ROOT_SCAN_FILE_CEILING and src/tensor_grep/cli/bootstrap.py's _search_paths_include_oversized_implicit_root — one constant, two Python call sites, so a future change to the ceiling cannot silently desync the Typer-app path from the flag-less bootstrap-passthrough path. The Rust rust_core/src/rg_passthrough.rs keeps its own copy of the same numeral, synced by convention (not a shared build-time constant across the Python/Rust boundary) — if you ever change the Python value, grep rg_passthrough.rs for the matching literal and update it in the same PR, or the two front doors will silently disagree on where the ceiling sits.
• This is a distinct axis from TG_DIR_SCAN_MAX_ENTRIES (env-configurable, a different directory- scan bound) — do not conflate the two when reading a scan-refusal report; check which constant/env var actually produced the observed refusal before describing the mechanism.

LSP provider

Provider modes: native / lsp / hybrid

The --provider flag appears on every symbol/navigation command (defs, refs, source, impact, callers, blast-radius*, context-render, edit-plan, agent, lsp) with the same three-way contract everywhere, default native:

tg defs REPO_PATH SYMBOL --provider lsp
tg blast-radius REPO_PATH SYMBOL --provider hybrid
tg lsp --provider hybrid

• native — tg's own tree-sitter/AST-derived symbol graph. Production default.
• lsp — routes through an external language server (ExternalLSPProviderManager). EXPERIMENTAL.
• hybrid — combines native with LSP evidence when available.

tg lsp validates the value explicitly and exits 2 on anything else ({"native", "lsp", "hybrid"} check, main.py:9656-9785):

Unsupported LSP provider mode; expected one of: native, lsp, hybrid

LSP-availability is not LSP-proof — this is a load-bearing distinction from AGENTS.md:163: "Treat tg lsp-setup / tg doctor --with-lsp availability as install evidence only; provider-backed navigation must report health_status, health_check, lsp_proof, lsp_evidence_status, and not_lsp_proof_reason when it falls back to native evidence. A navigation row counts as LSP proof only when it carries lsp_provider_response = true from a completed provider request." Do not tell a user "LSP is working" because tg doctor --with-lsp found a binary on PATH.

Production vs EXPERIMENTAL (default-OFF), and the guard that keeps it off

tg inventory: walk-only repo manifest (v1.19.0, #343)

tg inventory PATH [--json] [--max-repo-files N] [--deadline SECONDS] (src/tensor_grep/cli/inventory.py, registered main.py:8292-8404) emits a single-pass file/byte/language/category manifest by reusing the same gitignore-aware walker (repo_map._iter_repo_files) that orient/callers/ blast-radius trust — so counts stay truth-consistent with every other tg command and inherit its .tensor-grep/.git/vendor exclusions for free.

--max-repo-files defaults to 50_000, still well above the AST map limit — this is a deliberate, documented divergence, not an oversight. The AST-side number changed underneath this divergence (backlog #1, 2026-07-06): DEFAULT_AGENT_REPO_MAP_LIMIT was raised from 512 to 2000 (repo_map.py:157), and the CLI-side mirror _DEFAULT_AGENT_REPO_SCAN_LIMIT (main.py:82) was raised to match — do not describe the AST cap as 512 anymore.

• DEFAULT_MAX_INVENTORY_FILES = 50_000 (inventory.py:40), passed to the CLI option as a literal 50_000 (grep -n "50_000" src/tensor_grep/cli/main.py -- no line range: the old :8406-8413 pin sat INSIDE a --deadline option block that the 2026-08-23 de-duplication deleted outright, so it has no successor line to re-stamp to) rather than importing the constant, so the (heavy) repo_map import stays lazy. A nearby code comment still says "matching map's 512 pattern" — that comment is about the STYLE (keep-literal, don't import), not the current live number; map's own limit is 2000 now, not 512. A guard test pins the 50_000 literals together; re-verify with grep -rn "50_000" src/tensor_grep/cli/inventory.py src/tensor_grep/cli/main.py.
• DEFAULT_AGENT_REPO_MAP_LIMIT = 2000 (repo_map.py:157) budgets a full AST parse per file for tg map/orient/context/edit-plan/session repo-map defaults — reusing it for inventory would silently truncate any repo over ~2000 files and defeat the "whole-repo manifest" purpose (inventory.py:36-39 states this explicitly in a code comment). inventory is walk-only (stat() + an 8KB read for binary-sniffing per file), orders of magnitude cheaper than an AST parse, so a much higher cap (50_000) is still safe even after the AST-side raise.
• CALLER_SCAN_FILE_CEILING was ALSO raised, 512→2000 (repo_map.py:167-177; backlog #57, 2026-07-09) — the "DIFFERENT constant that stays at 512" framing this file previously used is itself now stale. It remains logically separate from DEFAULT_AGENT_REPO_MAP_LIMIT (they just now happen to share a value) — the raise was safe only because #478 had already threaded a --deadline hard-bound through the caller-scan loop, closing the task #52 100s-hang risk ("100s on a 1941-file repo at the old 512 cap", repo_map.py:163-164) that originally kept this ceiling frozen below the map default (repo_map.py:1638-1642). It still backstops the flag-less (--deadline omitted) default path and a --max-repo-files-raised mega-repo; raising past 2000 needs fresh cost data (repo_map.py:167-177). If you see the bare number 512 anywhere in this subsystem going forward, it is describing HISTORY — check which constant before assuming either reading is still live.
• Truncation is never silent, and it now has two distinguishable causes (scan_limit.truncation_cause in the JSON payload): "project-files" when --max-repo-files' count cap was hit, or "deadline" when --deadline (below) fired first; the key is always present, null when the scan completed. Text output mirrors the split (inventory.py:348-359): [!] truncated at max_files=... (cause=project-files) vs. [!] stopped after the time budget (cause=deadline) — both ASCII-only (fixed from a U+26A0 emoji that crashed typer.echo on Windows cp1252 consoles — #346, commit 6b7b518; ASCII-only is now the rule for all tg CLI output, not just inventory). Either cause trips the same shared exit-2 gate, _scan_incomplete (main.py:10817, checked at main.py:8329-8447) — it only looks at possibly_truncated, not which cause fired.
• Fails closed: a nonexistent path raises FileNotFoundError -> CLI exits 1 (inventory.py:201-202) — a missing path must never read as a valid empty repo.

--deadline SECONDS: the wall-clock twin of --max-repo-files (registered main.py:8297-8423)

Threads a deadline_seconds float (inventory.py:187) into build_inventory() so a huge/slow tree returns a partial, honestly-labeled manifest instead of hanging. inventory's own --deadline predates and is unrelated to #585 (2026-07-14), which extended --deadline to source/docs-coverage/blast-radius-plan instead (a disjoint set of commands) — inventory's flag shipped earlier (#395, issue #53), was hardened into a true wall-clock bound by #478 (issue #52), then had a zero-count bug fixed (next bullet) by #516.

• The walk and the per-file loop split the budget, they do not share it (inventory.py:204-223; _WALK_PHASE_DEADLINE_FRACTION = 0.7, inventory.py:48) — the in-source comment calls this the "#130(a) fix" (shipped as #516 per commit history). The walk phase (_iter_repo_files) gets only the first 70% of deadline_seconds; the remainder is reserved for the per-file stat()/binary-sniff loop. Before this split, a slow walk could consume the entire deadline, so the per-file loop's very first deadline check fired immediately and totals.files read 0 despite the walk having discovered real files — the in-source comment names the repro outright (inventory.py:204-208: "the 76s dogfood gap on tg inventory --deadline 30").
• Either phase running out of budget sets truncation_cause = "deadline" (inventory.py:297-306) — including when the walk was ALSO count-capped by --max-repo-files; "deadline" wins the label because a longer --deadline would help where a higher --max-repo-files would not. An earlier version of this same guard mislabeled a real 20s deadline hit on C:/dev/projects as a file-cap truncation (inventory.py:303-304, dogfood 2026-07-05) — exactly the bug this two-cause split exists to prevent from recurring.
• Known narrow gap (low-priority, not load-bearing): the walk's initial listing of the SCANNED ROOT itself is not deadline-interruptible. _iter_repo_files (grep -n "^def _iter_repo_files" src/tensor_grep/cli/repo_map.py -- :1183 as of 2026-08-14, was :1143), on the max_files is not None branch inventory always takes (it always calls with max_files=max_files + 1, inventory.py:231-236), does one eager, unconditional entries = list(os.scandir(normalized_root)) (repo_map.py:1009-1010) before any deadline check exists in that branch — every subsequent bucket pull IS deadline-checked (repo_map.py:1052-1057), just not this one root-level call. On a root whose own immediate directory listing is itself pathologically large/slow, --deadline cannot preempt it. Dogfooded against a 300k+-file workspace-union tree: tg inventory --deadline holds up fine per-project and on most roots — this edge needs a pathologically huge flat fan-out sitting directly at the scanned path to trigger, and is not worth a load-bearing lazy-scandir rewrite on its own; see tensor-grep-large-repo-scale-campaign for the broader deadline-scale work this sits alongside.

Registration follows the standard 4-site table (KNOWN_COMMANDS in commands.py, native Rust Commands::Inventory in rust_core/src/main.rs, PUBLIC_TOP_LEVEL_COMMANDS in tests/e2e/test_routing_parity.py, the @app.command() in main.py) — see tensor-grep-architecture-contract for why each site exists.

# Re-verify the two caps and why they differ
grep -n 'DEFAULT_MAX_INVENTORY_FILES\|max_repo_files' src/tensor_grep/cli/inventory.py src/tensor_grep/cli/main.py
grep -n 'DEFAULT_AGENT_REPO_MAP_LIMIT = \|CALLER_SCAN_FILE_CEILING = ' src/tensor_grep/cli/repo_map.py

# Re-verify --deadline registration + the walk/per-file budget split
grep -n '"--deadline"' src/tensor_grep/cli/main.py
grep -n 'deadline_seconds\|_WALK_PHASE_DEADLINE_FRACTION' src/tensor_grep/cli/inventory.py

# Smoke-test the command against the real binary (not CliRunner)
tg inventory . --json | python -m json.tool | head -30

Checklist: adding a flag or command

This is the single highest-value thing to get right in this repo — miss a registration site and the new flag/command misroutes silently, passing CliRunner tests while breaking the real published binary. See tensor-grep-architecture-contract for the full 4-site command / 2-site search-flag registration table and the rationale for why each site exists, and tensor-grep-change-control for the PR/merge gate around it (AGENTS.md:178-196) — this skill does not restate that table.

Worked non-registration example — tg prepare --out FILE (v1.93.0/#705, A12(d)). Not every new flag triggers all 3 registration concerns this skill and its siblings track — a useful example of "none of the above applied" to calibrate against: adding --out to the already-registered tg prepare command needed (1) no new 4-site command registration (the command already existed), (2) no 2-site search-flag registration (--out is not a search flag — it never reaches bootstrap's rg-passthrough front door), and (3) no SearchConfig field-coverage classification (prepare doesn't build a SearchConfig at all). It was a same-command, same-registration-footprint addition — a new typer.Option on an existing @app.command, nothing else. Recognizing when a change genuinely needs none of the 3 checklist items (vs. assuming every new flag does) saves a wasted registration audit.

The third checklist item: native-delegation field coverage (SearchConfig)

Adding a new field to SearchConfig (src/tensor_grep/core/config.py) is a third registration concern, separate from the 4-site command table and the 2-site search-flag table above — and it is easy to miss because it fails silently, not loudly.

Why it exists: _can_delegate_to_native_tg_search (grep -n "^def _can_delegate_to_native_tg_search" src/tensor_grep/cli/main.py -- :4095 as of 2026-08-14, was :3709) hands an entire search off to the native tg subprocess, which sys.exit()s before the Python-side BM25 rerank and the in-backend file sort ever run. Any SearchConfig field that is output-affecting but neither forwarded into the native argv (_build_native_tg_search_command -- grep -n "^def _build_native_tg_search_command" src/tensor_grep/cli/main.py, :4117 as of 2026-08-14, was :3731) nor listed in the refuse-tuple _NATIVE_TG_DELEGATION_DEFAULT_REQUIRED_FIELDS (grep -n "_NATIVE_TG_DELEGATION_DEFAULT_REQUIRED_FIELDS = " src/tensor_grep/cli/main.py -- :1966 as of 2026-08-14, was :1894 onward) gets silently dropped — the search still runs and returns a result, just the wrong one (unranked/unsorted), which is worse than a crash because suppression reads as absence. This is the same bug class as the -u/-uu no-op fixed in #336; the receipt this time was #342 (commit 5e6f780, v1.18.6->v1.19.0 range): rank_bm25 and sort_files were parsed but forwarded nowhere, so tg search --rank --cpu / --sort-files --cpu silently returned unranked/unsorted output on the delegated fast path.

The gate is now a governance ratchet, not a convention — tests/unit/test_native_delegation_field_coverage.py AST-derives the forwarded-field set directly from _build_native_tg_search_command's source (via ast.walk over `config.<

…(truncated)