Parallel Approved Issue Worktrees

Parallel Approved Issue Worktrees

Use this when:

• multiple GitHub issues are already status:plan-approved
• the streams are file-disjoint or can be serialized by ownership waves
• you want real implementation work to start now
• delegate_task workers are too unreliable/slow for repo writes or time out

Why this exists

In live use, a well-designed approved execution wave still failed when launched through delegate_task subagents: both workers timed out before returning usable summaries. The reliable fallback was to run Codex directly in isolated git worktrees with committed prompt artifacts and local plan-approval markers.

This pattern gives:

• real repo writes in the intended checkout
• explicit zero git contention
• auditable prompt artifacts
• easier monitoring/recovery than delegated hidden sandboxes

Core pattern

1. Commit the execution pack first

Before launching workers, commit and push the shared execution pack on main so every worktree can read the same immutable prompt files.

Typical artifacts:

• docs/plans/execution-packs/<date>-approved-<wave>/master-plan.md
• docs/plans/execution-packs/<date>-approved-<wave>/execution-readme.md
• docs/plans/execution-packs/<date>-approved-<wave>/stream-<issue>.md

2. Create one worktree per parallel stream

Example:

git worktree add -b issue-335-exec /mnt/local-analysis/worktrees/repo-335 origin/main
git worktree add -b issue-338-exec /mnt/local-analysis/worktrees/repo-338 origin/main

Rules:

• one issue/stream per worktree
• each worktree gets its own branch
• do not share owned files across concurrently running worktrees
• before creating a new worktree, check whether a clean existing issue-specific worktree already exists and can be reused

Live-use recovery rule:

• git worktree add ... can time out during the checkout/update phase even though the worktree was actually created successfully.
• If the add command times out, do NOT blindly retry with the same path/branch.
• First inspect:
  • git worktree list
  • whether the target directory now exists
  • git status --short, branch name, and HEAD inside that target worktree
• If the worktree exists and is clean, reuse it.
• Only remove/recreate it if the checkout is incomplete or dirty in a way you cannot trust.

Corrupt-worktree fallback learned in live use:

• A timed-out git worktree add can leave a target directory and branch behind while the git metadata under .git/worktrees/<name> is missing/corrupt. Symptom: fatal: not a git repository: .../.git/worktrees/<name> when running git worktree list or git status inside the target.
• Recovery sequence:
  1. Confirm the target directory exists and the branch exists: test -d <path> and git branch --list <branch>.
  2. Remove the broken target directory: rm -rf <path>.
  3. Delete the leftover branch if it only came from the failed worktree attempt: git branch -D <branch>.
  4. If repeated git worktree add is slow/risky in a very large repo, use an isolated shared clone instead: git clone --shared --branch main <repo-url> <path>.
  5. In the clone, create/commit the local .planning/plan-approved/<issue>.md marker before launching the worker.
• Treat the shared clone as equivalent isolation for a single issue lane: still use absolute prompt/log paths, a dedicated branch before push, and the same owned/read-only/forbidden path contract.

3. Add local plan-approved markers inside each worktree

If local hooks enforce .planning/plan-approved/<issue>.md, create and commit the marker in the exact worktree that will perform writes.

Example:

mkdir -p .planning/plan-approved
printf 'Issue #335 plan approved via GitHub label by user on YYYY-MM-DD.\n' > .planning/plan-approved/335.md
git add .planning/plan-approved/335.md
git commit -m "chore(planning): approve issue #335 for execution"

Why:

• GitHub status:plan-approved alone may not satisfy local hooks
• the marker must exist in the same checkout that writes files
• committing it avoids freshness/self-approval gate failures mid-run

4. Post GitHub execution-start comments

For each stream, post a concise comment describing:

• wave/stream name
• isolated worktree execution
• zero-overlap guarantee
• whether another stream is running in parallel

5. Launch Codex directly in each worktree

5. Launch Codex directly in each worktree

Use the committed stream prompt file as the positional prompt argument, not stdin.

Example:

PROMPT=$(< /path/to/repo/docs/plans/execution-packs/<wave>/stream-335.md)
cd /path/to/.worktrees/repo-335
Codex -p \
  --permission-mode acceptEdits \
  --no-session-persistence \
  --output-format text \
  --max-turns 80 \
  "$PROMPT" </dev/null | tee .Codex-stream-335.log

Critical launch guardrail learned in live overnight runs:

• when launching from worktrees, use absolute paths for both the prompt file and the log file in the shell command, even if the process workdir is already set to the target worktree
• create the log directory with an absolute path too when the worktree is nested or when tee writes outside the immediate cwd assumptions
• if you use a relative path like .planning/quick/stream.md and the shell reports No such file or directory even though the file exists, relaunch with /absolute/path/to/.planning/quick/stream.md
• similarly, a lane can complete successfully while tee exits non-zero because the target log path was relative/nonexistent; treat that as launcher/logging failure first, not task failure

Safer pattern:

PROMPT=$(< /absolute/path/to/worktree/.planning/quick/stream.md)
mkdir -p /absolute/path/to/worktree/logs
Codex -p \
  --permission-mode acceptEdits \
  --no-session-persistence \
  --output-format text \
  --max-turns 80 \
  "$PROMPT" </dev/null | tee /absolute/path/to/worktree/logs/stream.log

Important launch hardening learned in live use:

• Prefer absolute paths for both the prompt file and the log file when launching unattended background Codex runs from worktrees.
• A relative prompt read like .planning/quick/foo.md can fail with No such file or directory even when the file exists in the worktree, especially when the launcher shell/session context is not exactly what you expect.
• Safer pattern:

PROMPT=$(< /absolute/path/to/worktree/.planning/quick/stream-335.md)
Codex -p ... "$PROMPT" </dev/null | tee /absolute/path/to/worktree/logs/stream-335.log

• After launch, immediately poll the tracked process once. If it exits instantly with a missing-prompt error, relaunch with absolute paths rather than debugging the worktree contents first.

Critical launch detail learned in live overnight use:

• When launching from background shell commands inside worktrees, prefer absolute prompt-file paths in the PROMPT=$(< ...) substitution instead of relative paths like .planning/quick/....
• We observed a failure mode where the prompt file definitely existed in the target worktree, but the background shell still errored with No such file or directory when using the relative path. Re-launching with the absolute path to the same file succeeded immediately.
• Recommended pattern:

PROMPT=$(< /absolute/path/to/worktree/.planning/quick/issue-335-prompt.md)
Codex -p --permission-mode acceptEdits --no-session-persistence --output-format text --max-turns 80 "$PROMPT" </dev/null | tee /absolute/path/to/worktree/logs/issue-335.log

• After launch, poll the process once. If it exits quickly with a missing-prompt error, retry immediately with absolute prompt and log paths before assuming the worktree or prompt generation failed.

Worktree-local prompt pattern:

PROMPT=$(< /absolute/path/to/.planning/quick/issue-335-overnight-prompt.md)
Codex -p --permission-mode acceptEdits --no-session-persistence \
  --output-format text --max-turns 80 "$PROMPT" </dev/null | tee logs/issue-335.log

Why:

• in live overnight worktree launches, relative prompt paths failed with No such file or directory even though the prompt files existed in the target worktree
• switching to absolute prompt paths fixed the launch immediately

Recommended flags:

• --permission-mode acceptEdits
• --no-session-persistence
• --output-format text
• close stdin with </dev/null>
• log to a worktree-local file

6. Monitor and serialize any conflicting follow-on stream

If a later stream shares a file/path with an active stream, keep it queued.

Typical pattern:

• Wave 1: run streams A and B in parallel because they are disjoint
• Wave 2: run stream C only after A finishes because A and C both own src/.../__init__.py

6.5 Log-path guard for nested repos and deep worktrees

When launching unattended Codex runs with tee, create the exact target log directory using an absolute path before the run starts.

Observed failure mode:

• the worker command used mkdir -p logs && ... | tee /abs/path/to/logs/run.log
• in a nested repo/deep worktree, the absolute .../logs/ directory did not already exist even though relative logs/ did
• tee failed with No such file or directory, the overall process exited non-zero, but the Codex run itself still completed substantial work

Safe launch pattern:

mkdir -p /abs/path/to/worktree/logs
PROMPT=$(< /abs/path/to/prompt.md)
Codex -p \
  --permission-mode acceptEdits \
  --no-session-persistence \
  --output-format text \
  --max-turns 80 \
  "$PROMPT" </dev/null | tee /abs/path/to/worktree/logs/run.log

Operational rule:

• treat tee/log-path failures separately from task success
• if the process exits non-zero but the terminal output or GitHub comment shows the task completed, classify it as a launcher/logging failure first, not an implementation failure
• after any non-zero exit, inspect the worktree state and issue comments before rerunning the lane

Handoff template

Each stream prompt should include:

• issue number and URL
• owned paths
• read-only paths
• forbidden paths
• exact TDD targets
• regression boundary tests
• implementation rules
• closeout requirements
• explicit instruction not to close the issue

Monitoring checklist

For each running worker:

• check process liveness
• check target worktree git status --short
• check whether expected test files/source files appear
• treat empty logs cautiously; Codex logs can stay buffered for a while
• if a worker times out or stalls, inspect the worktree before killing it

Pre-dispatch live-state filter (important)

Before launching an approved-issue overnight wave, verify each candidate issue is still a real implementation target right now.

Required checks per issue:

• issue is still OPEN (gh issue view <n> --json state,labels)
• issue still has status:plan-approved
• local .planning/plan-approved/<issue>.md marker exists in the exact worktree/checkouts that will write
• issue is not already satisfied by an existing branch/commit/worktree outcome
• owned paths are still disjoint from the other selected streams
• if the issue is a prerequisite contract for child issues, verify the contract's exact locked fields on origin/main before launching children; do not trust child status:plan-approved labels when local plans still contain stale prerequisite/deferred-path language

Practical lesson from live use:

• an issue can remain in your approved pool assumptions while already being CLOSED or already effectively implemented on its dedicated branch/worktree
• in that case, do not waste a write-capable lane on reimplementation; instead run the stream in verification-first mode and post a proof comment if appropriate
• when selecting 4 parallel lanes, expect some approved issues to collapse into already done / verify only outcomes rather than producing fresh diffs
• after a prerequisite contract lands, run a read-only child-plan reconciliation wave before implementation; classify each child as READY_NOW, READY_AFTER_PATCH, or BLOCKED, and patch stale plan assumptions such as conditional operator-map host paths, deferred registry filenames, or scorecard-as-authority language before launching write lanes

Dispatcher/executor drift and sandbox-scope check

A background Codex lane can exit 0 without touching the intended isolated clone when the spawned session is sandboxed to a different root. The worker may correctly detect a concurrent completion and refuse duplicate work, but the dispatcher must still verify where the real commit landed.

Before treating a lane as landed:

• read the worker log for duplicate-parallel-completion, sandbox denial, or "no files changed" language
• verify the claimed commit with git show --stat --name-only <sha>
• verify which branch contains it: git branch -a --contains <sha>
• verify whether it reached origin/main: git merge-base --is-ancestor <sha> origin/main; echo $?
• if it landed on an integration branch rather than main, cherry-pick the validated commit(s) into a clean main-line clone/worktree, re-run targeted tests there, then push HEAD:main
• if the contract was missing an exact decision needed by children, make a tiny follow-up commit that locks the decision and updates the test to assert it before starting child work

For future isolated-clone dispatches, ensure the worker's allowed/sandbox directories include the intended clone path (for Codex, use the equivalent of --add-dir when available) or launch directly from a scope-compatible checkout. Otherwise the prompt may name a path the worker cannot write.

Post-run recovery when Codex exits on max-turns

If a background Codex worker exits with Error: Reached max turns (...), do not treat the run as a total failure.

Recovery sequence:

1. inspect the worktree immediately:
   • git status --short
   • git log --oneline -5
   • git show --stat --oneline -1
2. determine whether the worker already produced a useful local commit or partial diff
3. run the targeted validation commands yourself from the orchestrator session
4. run an independent adversarial review on the produced diff/commit
5. then choose exactly one outcome:
   • landable: proceed to commit/push/comment flow if validation + review pass
   • blocked: post a GitHub blocker comment summarizing the fresh validation/review findings and leave the issue open
   • no-op: if nothing meaningful changed, treat as failed attempt and relaunch or retire the lane

Practical lesson from live use:

• a max-turns failure can still leave behind a nearly complete local implementation commit
• the correct response is central verification + review, not automatic discard or automatic merge

Local allowlist / permission fallback

In some repos, Codex can edit files but cannot run the exact validation or GitHub commands you asked for because repo-local settings block commands like:

• pytest
• uv run
• python -m pytest
• gh

When that happens, do not discard the worker run.

Use this split-responsibility pattern instead:

1. let the worker finish the implementation inside its isolated worktree
2. read the worker log and inspect the changed files yourself
3. run the planned validation centrally from the parent/orchestrator session in that worktree
4. commit and push from the orchestrator if the worker could not
5. post GitHub validation and closeout comments from the orchestrator

This preserves the implementation value of the worker while moving blocked shell actions to the session that actually has the needed tool permissions.

Recovery pattern

If a worker stalls:

1. inspect target file diffs in the worktree
2. inspect git status --short
3. if intended changes are already present, run the planned tests manually
4. commit/push yourself if the code is sound
5. post the GitHub summary manually

Nested-repo / cross-repo target guard

When a workspace-hub issue asks for changes in a tier-1 repo such as digitalmodel, do not assume a workspace-hub worktree contains the nested repo or that the worker will write to the intended git repository.

Before launching write-capable workers:

• verify the actual target repo root with git -C <target> rev-parse --show-toplevel
• verify the active branch in the target repo, not just in workspace-hub
• if the target repo is a separate checkout/submodule/nested repo, create the implementation branch/worktree in that target repo directly
• include the target repo's absolute path in the prompt and validation commands
• after worker completion, run git status --short, git log --oneline -3, and git diff --stat origin/main...HEAD in the target repo, not the parent workspace

Clean PR branch guard:

• before creating a PR, inspect gh pr view or git log origin/main..HEAD to ensure the branch contains only the intended commits
• if a PR accidentally includes an unrelated prior commit, close it immediately with an explanatory comment
• create a clean branch from origin/main, cherry-pick only the validated intended commit(s), re-run targeted tests on the clean branch, push, and open a replacement PR
• post correction comments to the tracking issues with the clean PR URL and superseded PR URL

This prevents a successful implementation wave from producing a polluted PR when the active target repo branch already had unrelated pending work.

When not to use this pattern

• shared-file changes without a clean serialization plan
• issues that are not already plan-approved
• streams that depend on unresolved decisions from each other
• work requiring user interaction mid-run

Practical note

This is a better fit than delegate_task when the real goal is implementation in the actual repo, especially for approved issue waves where correctness depends on the exact checkout state, hook behavior, and git ownership boundaries.