overnight-worktree-Codex-noop-recovery

Overnight worktree Codex no-op recovery

Use this when an unattended multi-worktree Codex batch is launched correctly, exits with code 0, but expected planning/report artifacts are missing.

Trigger pattern

• Background Codex -p jobs in separate worktrees
• process poll shows exit_code: 0
• stdout/log output is empty or unhelpful
• expected result files under docs/reports/ / docs/plans/ / scripts/review/results/ were not created

What happened in the learned case

A 3-worktree overnight planning wave was launched safely from isolated worktrees. All three Codex runs exited successfully, but two produced no expected outputs and one only preserved pre-existing artifacts. The initial prompts were too easy for Codex to effectively no-op while still returning success. A first rerun also failed because the launcher used relative prompt-file paths that the shell did not resolve as expected.

Recovery workflow

1. Verify the run really no-op'd

• Check the expected artifacts directly, not just process exit code.
• For each worktree, inspect:
  • required summary/report file
  • target docs/plans/*issue* files
  • target scripts/review/results/*issue* files
• If none exist, treat the run as failed even if Codex exited 0.

2. Keep the worktree isolation

• Do NOT rerun in the dirty main checkout.
• Reuse or create one worktree per terminal/lane.
• Preserve zero-overlap write boundaries.

3. Harden the prompt before rerunning

• Make the FIRST mandatory action the creation of a unique summary/result file.
• Require a STARTED section immediately.
• Require a COMPLETED or BLOCKED section before exit.
• State explicitly: do not exit without writing this file.
• Reduce ambition: even if deeper plan edits fail, the summary file must still be produced.

Example requirement block:

• docs/reports/2026-04-23-terminal-N-...summary.md
• MUST include STARTED timestamp
• MUST include issues in scope
• MUST include blockers
• MUST append COMPLETED/BLOCKED before exit

4. Use absolute prompt paths in the launcher

• Avoid PROMPT=$(< docs/plans/.../prompt.md) in reruns.
• Prefer:
  • PROMPT=$(< /abs/path/to/prompt.md)
• This removes cwd/path ambiguity in copied worktree setups.

5. Relaunch as background jobs

• Same Codex -p --permission-mode acceptEdits --no-session-persistence ...
• Keep one process per worktree.
• Poll after launch to ensure the shell found the prompt file.

6. Morning interpretation rule

• exit 0 + no artifacts = failed/no-op run
• exit nonzero due to prompt path = launcher failure
• only artifact existence counts as success

Recommended launch pattern

wt=/mnt/local-analysis/worktrees/ws-tier1-knowledge-overnight-t2
prompt=$wt/docs/plans/overnight-prompts/2026-04-22-tier1-knowledge-beef-up/rerun-terminal-2-engineering-core-and-utilities.md
cd "$wt"
mkdir -p logs
PROMPT=$(< "$prompt")
Codex -p \
  --permission-mode acceptEdits \
  --no-session-persistence \
  --output-format text \
  --max-budget-usd 20 \
  "$PROMPT" </dev/null > logs/Codex-tier1-terminal-2-rerun.log 2>&1

Checks to run after a suspicious success

python - <<'PY'
from pathlib import Path
root=Path('/path/to/worktree')
for pat in [
    'docs/reports/expected-summary.md',
    'docs/plans/*2461*',
    'scripts/review/results/*2461*',
]:
    print(pat, list(root.glob(pat)))
PY

Key lesson

For unattended Codex planning batches, prompt design must force an observable artifact very early. Process exit codes and silent logs are not reliable proof of work completion.