Test Evidence Lives on an Orphan Branch, Never in the Code Branch
- The test step always collects evidence OUTSIDE the worktree, in the directory the executor resolved once as
StepContext.EvidenceDir; nothing stages or commits it into the pushed branch, so evidence can never reach the default branch's history. Withtest.evidence.store_in_repoand a derivable GitHub link base, the PR step callspublishRunEvidence(internal/pipeline/steps/evidence_publish.go), which copies the directory onto the push-target repo's orphan evidence branch throughinternal/evidenceand hands the PR body its links. A provider without derivable links does not push the branch. internal/evidenceowns the mechanism and its fail-closed rules: plumbing only (scratchGIT_INDEX_FILE+hash-object/write-tree/commit-tree), so HEAD, the index, and the worktree are untouched and a detached or shallow clone works; the parent is the just-fetched remote tip so the push is a plain fast-forward and never a force; an existing branch without the.no-mistakes-evidencemarker at its tip is refused, which is what makes a wrong branch name (main) harmless. Every failure returns an error and the PR body falls back to local-path references rather than links that would not resolve.- PR links are pinned to the evidence COMMIT, not the branch, so a later run overwriting the same paths cannot change what an old PR shows. Link bases come from
Repo.UpstreamURL/ForkURL, never the push URL, which can carry a credential. test.evidence.branchis trusted-only inEffectiveRepoConfig(it names a ref the daemon pushes to);local_root/retention/max_runsare global-only (applyEvidenceStorageOverridesis called fromMergewithGlobalConfigalone); the rest oftest.evidencestays pushed-readable. Invalid branch names, relativelocal_root, unparseableretention, and negativemax_runsall fail the config at parse time (validateTestRaw).- Regressions:
internal/evidence/publish_test.go,internal/evidence/branch_test.go,internal/pipeline/steps/evidence_publish_test.go,TestPushStep_DoesNotPublishTestEvidenceIntoThePushedBranch,TestEffectiveRepoConfig_EvidenceBranchTrustedOnly,TestLoadGlobalConfig_InvalidEvidenceBranchFailsClosed,internal/config/evidence_storage_test.go.
no-mistakes Owns Its Own Scratch (never the shared system temp dir)
- Evidence lives at
<NM_HOME>/evidence/<runID>(paths.EvidenceDir/EvidenceRoot/RunEvidenceDir), neveros.TempDir(). The daemon's service unit exports only HOME, PATH, and proxy vars, soTMPDIRis unset andos.TempDir()resolved to the shared/tmp- a systemd tmpfs on Ubuntu 24.10+, so evidence consumed RAM. The app root is disk-backed on all three platforms, so there is deliberately NOruntime.GOOSbranch; do not add one. - One owner for the path: the executor resolves it (
Executor.runEvidenceDir) intoStepContext.EvidenceDir, andagent.WithSteering(a, evidenceRoot)takes it as an argument. Steps and the steering preamble must never rebuild it - two independentos.TempDir()copies is exactly the drift this replaced. - Cleanup is ours, in three layers:
RunManager.cleanupRunEvidenceremoves a finished run's dir when empty (os.Remove, neverRemoveAll- the test step creates the dir before the agent decides it has anything to write, and that litter was 94% of observed accumulation),reapEvidencebounds the directory by age and count oldest-first, andreapLegacyEvidencedrains the pre-relocation temp directory under the same policy. All three reuseskipWorktreeCleanup's pending/running guard and are best effort. No OS temp timer is load-bearing. - HELD SCOPE:
internal/eval/replay.gosandboxes stay in the system temp directory. They are the largest scratch this program creates, but a replay materializes its own nested NM_HOME and worktree whileStore.Prune, the case records, and the object pools all live under<NM_HOME>/eval- so relocating the sandbox inside the app root nests it in the state it is replaying, which e2eTestEvalJourneyrefuses on purpose. Moving it needs a disk-backed root outside NM_HOME, which does not exist yet; do not "fix" it by weakening that assertion. Every remainingos.MkdirTemp("", ...)caller is auto-named and self-cleaning withdefer; keep it that way. - Regressions:
internal/paths/evidence_test.go,internal/config/evidence_storage_test.go,internal/daemon/evidence_reap_test.go,TestSteeringNamesTheConfiguredEvidenceRoot,TestTestEvidenceDir_DefaultResolutionStaysUnderTheAppRoot, e2eTestTestEvidenceLivesUnderAppRootNotSharedTemp/TestRunCleanupLeavesNoEmptyEvidenceDirectory.