# Test Evidence Storage

> Use when changing test evidence collection, publication, paths, retention, or scratch-directory cleanup.

- Skill: `mvanhorn/test-evidence-storage` (Agent Skill)
- Install (CLI): `npx skillmds@latest add mvanhorn/test-evidence-storage`
- Raw SKILL.md: https://api.skillmd.com/api/skills/mvanhorn/test-evidence-storage/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- Author: mvanhorn (https://skillmd.com/u/mvanhorn)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/mvanhorn/test-evidence-storage

---


**Test Evidence Lives on an Orphan Branch, Never in the Code Branch**

- The test step always collects evidence OUTSIDE the worktree, in the directory the executor resolved once as `StepContext.EvidenceDir`; nothing stages or commits it into the pushed branch, so evidence can never reach the default branch's history. With `test.evidence.store_in_repo` and a derivable GitHub link base, the PR step calls `publishRunEvidence` (`internal/pipeline/steps/evidence_publish.go`), which copies the directory onto the push-target repo's orphan evidence branch through `internal/evidence` and hands the PR body its links. A provider without derivable links does not push the branch.
- `internal/evidence` owns the mechanism and its fail-closed rules: plumbing only (scratch `GIT_INDEX_FILE` + `hash-object`/`write-tree`/`commit-tree`), so HEAD, the index, and the worktree are untouched and a detached or shallow clone works; the parent is the just-fetched remote tip so the push is a plain fast-forward and never a force; an existing branch without the `.no-mistakes-evidence` marker at its tip is refused, which is what makes a wrong branch name (`main`) harmless. Every failure returns an error and the PR body falls back to local-path references rather than links that would not resolve.
- PR links are pinned to the evidence COMMIT, not the branch, so a later run overwriting the same paths cannot change what an old PR shows. Link bases come from `Repo.UpstreamURL`/`ForkURL`, never the push URL, which can carry a credential.
- `test.evidence.branch` is trusted-only in `EffectiveRepoConfig` (it names a ref the daemon pushes to); `local_root`/`retention`/`max_runs` are global-only (`applyEvidenceStorageOverrides` is called from `Merge` with `GlobalConfig` alone); the rest of `test.evidence` stays pushed-readable. Invalid branch names, relative `local_root`, unparseable `retention`, and negative `max_runs` all fail the config at parse time (`validateTestRaw`).
- Regressions: `internal/evidence/publish_test.go`, `internal/evidence/branch_test.go`, `internal/pipeline/steps/evidence_publish_test.go`, `TestPushStep_DoesNotPublishTestEvidenceIntoThePushedBranch`, `TestEffectiveRepoConfig_EvidenceBranchTrustedOnly`, `TestLoadGlobalConfig_InvalidEvidenceBranchFailsClosed`, `internal/config/evidence_storage_test.go`.

**no-mistakes Owns Its Own Scratch (never the shared system temp dir)**

- Evidence lives at `<NM_HOME>/evidence/<runID>` (`paths.EvidenceDir`/`EvidenceRoot`/`RunEvidenceDir`), never `os.TempDir()`. The daemon's service unit exports only HOME, PATH, and proxy vars, so `TMPDIR` is unset and `os.TempDir()` resolved to the shared `/tmp` - a systemd tmpfs on Ubuntu 24.10+, so evidence consumed RAM. The app root is disk-backed on all three platforms, so there is deliberately NO `runtime.GOOS` branch; do not add one.
- One owner for the path: the executor resolves it (`Executor.runEvidenceDir`) into `StepContext.EvidenceDir`, and `agent.WithSteering(a, evidenceRoot)` takes it as an argument. Steps and the steering preamble must never rebuild it - two independent `os.TempDir()` copies is exactly the drift this replaced.
- Cleanup is ours, in three layers: `RunManager.cleanupRunEvidence` removes a finished run's dir when empty (`os.Remove`, never `RemoveAll` - the test step creates the dir before the agent decides it has anything to write, and that litter was 94% of observed accumulation), `reapEvidence` bounds the directory by age and count oldest-first, and `reapLegacyEvidence` drains the pre-relocation temp directory under the same policy. All three reuse `skipWorktreeCleanup`'s pending/running guard and are best effort. No OS temp timer is load-bearing.
- HELD SCOPE: `internal/eval/replay.go` sandboxes stay in the system temp directory. They are the largest scratch this program creates, but a replay materializes its own nested NM_HOME and worktree while `Store.Prune`, the case records, and the object pools all live under `<NM_HOME>/eval` - so relocating the sandbox inside the app root nests it in the state it is replaying, which e2e `TestEvalJourney` refuses on purpose. Moving it needs a disk-backed root outside NM_HOME, which does not exist yet; do not "fix" it by weakening that assertion. Every remaining `os.MkdirTemp("", ...)` caller is auto-named and self-cleaning with `defer`; keep it that way.
- Regressions: `internal/paths/evidence_test.go`, `internal/config/evidence_storage_test.go`, `internal/daemon/evidence_reap_test.go`, `TestSteeringNamesTheConfiguredEvidenceRoot`, `TestTestEvidenceDir_DefaultResolutionStaysUnderTheAppRoot`, e2e `TestTestEvidenceLivesUnderAppRootNotSharedTemp` / `TestRunCleanupLeavesNoEmptyEvidenceDirectory`.