Dedup

Dedup Loop (dedupe-only)

Input: a git base SHA/ref to dedupe against. Default/recommended is auto, which resolves a branch-wide merge-base from integration/default refs first, then falls back to upstream (@{u}).

Preconditions

• Confirm you are in the intended repo: git rev-parse --show-toplevel
• Ensure you have a branch-wide base commit SHA (the fork-point/common-ancestor with the integration branch).
• Start from a clean working tree (required):
  • git status --porcelain should be empty.
• Ensure git can create commits (stop if missing):
  • If commits fail due to missing identity, configure user.name and user.email.
• Ensure both CLIs are available (codex and claude) because reviewer/responder must be different vendors.

Scope / intent

• This is NOT a general code review.
• Only act on:
  • duplicate code introduced/expanded by BASE_SHA..HEAD, and/or
  • code introduced by BASE_SHA..HEAD that reimplements an existing shared util already in the repo.
• Do not apply unrelated fixes (style, perf, security, naming, error handling) unless strictly required to complete a dedupe/reuse change safely.

Expected runtime

Typically runs 10–30 minutes, but allow at least 60 minutes.

Do not interrupt/restart the subagent if it looks stuck. The loop script owns stuck/timeout handling and will exit on its own when it completes or when --timeout is reached. Prefer watching the periodic heartbeat output (--heartbeat-seconds) instead of tailing logs.

Why Claude sometimes looked “stuck”

Claude Code print mode can be silent in --output-format text until it finishes (including during tool work). This repo defaults Claude subprocesses to --output-format stream-json --include-partial-messages so the wrapper sees measurable progress and inactivity timeouts mainly trigger on true hangs.

Run

resolve_skill_dir() {
  local name="$1"
  local repo_root=""
  repo_root="$(git rev-parse --show-toplevel 2>/dev/null || pwd)"

  local candidates=(
    "$repo_root/.agents/skills/$name"
    "$repo_root/.claude/skills/$name"
    "$HOME/.agents/skills/$name"
    "$HOME/.codex/skills/$name"
    "$HOME/.claude/skills/$name"
  )

  for d in "${candidates[@]}"; do
    if [[ -d "$d" ]]; then
      echo "$d"
      return 0
    fi
  done

  echo "Error: skill '$name' not found in repo-scoped or user-scoped skill dirs." >&2
  return 1
}

DEDUP_SKILL_DIR="$(resolve_skill_dir dedup)"
python3 "$DEDUP_SKILL_DIR/scripts/run_dedup_loop.py" auto

Explicit branch-base example (if you want to pin it yourself):

INTEGRATION_REF="$(
  git symbolic-ref --quiet --short refs/remotes/origin/HEAD 2>/dev/null \
    || { git show-ref --verify --quiet refs/remotes/origin/main && echo origin/main; } \
    || { git show-ref --verify --quiet refs/remotes/origin/master && echo origin/master; } \
    || echo "@{u}"
)"
BASE_SHA="$(git merge-base --fork-point "$INTEGRATION_REF" HEAD 2>/dev/null || git merge-base "$INTEGRATION_REF" HEAD)"
python3 "$DEDUP_SKILL_DIR/scripts/run_dedup_loop.py" "$BASE_SHA"

Options:

• --max-iterations N (default: 1)
• --cli codex|claude (pin responder provider; reviewer is selected from provider pool and always differs from responder)
• --provider-pool codex,claude (provider pool for both subprocesses; reviewer/responder vendors are always distinct)
• --codex-model-pool ... and --claude-model-pool ... (supports model@effort)
• --progress-log <path> and --heartbeat-seconds N
• --artifacts-dir <path> (optional; stores reviewer outputs for responders to read)
• --timeout N (per-subprocess timeout in seconds; default: 3600)
• --review-only (responder does not apply fixes/commit)
• --scope <pathspec> (repeatable; restricts git diff to those paths)

Claude automation knobs (env vars; defaults shown):

• PROMPT_TEMPLATES_CLAUDE_OUTPUT_FORMAT=stream-json (text|json|stream-json)
• PROMPT_TEMPLATES_CLAUDE_MIN_VERSION=2.1.33 (fail fast if installed Claude Code is older)
• PROMPT_TEMPLATES_CLAUDE_STREAM_LOG_MAX_BYTES=10485760 (per invocation; set 0 for unlimited)
• PROMPT_TEMPLATES_CLAUDE_INACTIVITY_TIMEOUT_SECONDS=180 (set 0 to disable; in text/json mode it is disabled unless explicitly set)
• PROMPT_TEMPLATES_CLAUDE_INACTIVITY_MIN_RUNTIME_SECONDS=30
• PROMPT_TEMPLATES_CLAUDE_PROMPT_BUDGET_BYTES=0 (disabled by default; set >0 to enforce)

Behavior:

• Each iteration runs in two fresh subprocesses:
  • Reviewer subprocess: generates dedupe-only findings from BASE_SHA..HEAD (it fetches the diff locally via git diff).
  • Responder subprocess: agrees/rejects each finding; applies only agreed dedupe/reuse fixes; runs obvious checks; commits (it fetches the diff locally via git diff and reads reviewer output from the artifacts dir).
• The script enforces that reviewer and responder always run on different vendors (codex vs claude).
• Claude subprocesses run in a native PTY (when available) and have an inactivity timeout; on classifiable Claude automation failures the script retries once.
• If reviewer=claude fails after retry and responder is not pinned to codex, the script performs a role-swap fallback and re-runs the iteration with reviewer=codex,responder=claude (it prints a CLAUDE_FALLBACK: ... line when this happens).
• When the responder subprocess runs on Codex, it is invoked with --sandbox danger-full-access and -a never so it can run git commands (including committing fixes) without getting stuck on approvals or .git lock writes.
• In non---review-only mode, the script enforces commit hygiene per iteration: if responder leaves working-tree changes, it auto-commits them before the next iteration; if responder accepts findings but produces no committed changes, the loop stops.

Troubleshooting:

• --progress-log may include full tool outputs (diffs, file contents). Treat it as sensitive; stream-json logs are truncated by default via PROMPT_TEMPLATES_CLAUDE_STREAM_LOG_MAX_BYTES.