Codexloop

Codexloop

Drive a GitHub PR until Codex has no unresolved review comments, but do not cargo-cult its suggestions. Every comment is a claim to verify, not an instruction to obey. A wrong suggestion applied is worse than the comment itself.

How Codex differs from Greptile / Gemini

• No check-run, no score. Codex does not publish an X/5 confidence or a named check. It posts a PR review (state COMMENTED) plus inline review comments. Detection is by polling the reviews endpoint. "Satisfied" = zero unresolved comments (each either fixed or rebutted).
• Trigger phrase is @codex review. Codex reviews automatically when a PR opens or gets new commits if auto-review is enabled on the repo; the mention forces a fresh pass. Other useful mentions: @codex review focus on <area> for a scoped re-review.
• Priority, not confidence. Codex findings usually lead with a severity/priority word (P1, P2, P3 or critical/major/minor) or a short titled heading. Weight the top tier seriously; treat the bottom tier as usually-skippable nits unless clearly correct.
• Codex is terse and often reasons from the diff alone. Its characteristic failure is confidently asserting a bug based only on the changed hunk, without the surrounding file or the call sites. That makes "read the whole file before believing it" the single highest-value check.

Not for

• GitLab / Perforce (Codex code review is a GitHub app). For other review bots this catalog ships geminiloop, coderabbitloop, and qodoloop; for CI failures rather than review comments, use ci-fix-loop.

0. Resolve the Codex bot login (do this first, do not hardcode)

The connector's bot login varies by installation, so discover it from the PR rather than assuming it:

gh api repos/{owner}/{repo}/pulls/<PR>/reviews --paginate --jq '.[].user.login' | sort -u
gh api repos/{owner}/{repo}/pulls/<PR>/comments --paginate --jq '.[].user.login' | sort -u

Pick the login matching *codex* (commonly chatgpt-codex-connector[bot], sometimes codex[bot]) and export it as BOT.

Finding nothing here is NOT a stop condition. On a PR Codex has never reviewed there is no bot login to find yet: that is the normal starting state, not a missing app. When the probe comes back empty, fall through to step 2A, post @codex review, and re-run this probe once a review lands; until then match any login containing codex when polling. Only conclude the app is absent after step 2A's bounded wait has expired with no review and no codex-like login anywhere on the PR, and then say so and stop, rather than looping against nothing.

1. Identify the PR

gh pr view --json number,headRefName,headRefOid -q '{number,branch:.headRefName,head:.headRefOid}'

Switch to the PR branch if not already on it. Capture OWNER/REPO (gh repo view --json owner,name).

2. The loop (max 5 iterations)

Keep an explicit iteration counter and stop at 5: the cap is a real bound to enforce, not a figure of speech. Each pass through A–G is one iteration; on hitting the cap, go straight to the report and list what is still unresolved rather than starting a sixth.

A. Ensure a fresh Codex review on the current head

HEAD_SHA=$(gh pr view <PR> --json headRefOid -q .headRefOid)
# Only trigger if no Codex review already exists for this exact SHA:
HAVE=$(gh api repos/{owner}/{repo}/pulls/<PR>/reviews --paginate \
  --jq "[.[] | select(.user.login==\"$BOT\" and .commit_id==\"$HEAD_SHA\")] | length")
if [ "$HAVE" = "0" ]; then gh pr comment <PR> --body "@codex review"; fi

Poll for the review of THIS head to land. No check-run exists, so poll the reviews endpoint, and poll it on a deadline, never while true: a review that never arrives must end the skill with an honest timeout, not hang it.

# 10-minute deadline, one retry, then give up. DEADLINE/RETRIES are the
# enforcement of the bounds this skill claims: do not drop them.
wait_for_review() {                       # $1 = attempt label
  local deadline=$(( SECONDS + 600 ))
  while [ "$SECONDS" -lt "$deadline" ]; do
    R=$(gh api repos/{owner}/{repo}/pulls/<PR>/reviews --paginate \
      --jq "[.[] | select(.user.login==\"$BOT\" and .commit_id==\"$HEAD_SHA\")] | last")
    if [ -n "$R" ] && [ "$R" != "null" ]; then return 0; fi
    echo "waiting for Codex review of $HEAD_SHA ($1)..."; sleep 15
  done
  return 1
}

if ! wait_for_review "first wait"; then
  echo "no Codex review after 10m, retrying once"     # say the retry out loud
  gh pr comment <PR> --body "@codex review"
  if ! wait_for_review "after retry"; then
    echo "Codex did not review $HEAD_SHA after a retry; stopping and reporting."
    exit 1                                # honest timeout, never a success claim
  fi
fi

Codex can take several minutes on a large diff, which is why the deadline is generous. Report the retry in the final summary; two silent timeouts are the failure mode this guard exists to prevent.

B. Fetch the findings

• Summary: the review .body from the object above: read the overall take and the priority spread.
• Unresolved inline comments on the current head:

gh api repos/{owner}/{repo}/pulls/<PR>/comments --paginate \
  --jq ".[] | select(.user.login==\"$BOT\") | {id, path, line, body}"

Also pull the review threads + their resolved state via GraphQL (see step F) so you only act on unresolved ones.

C. Critically evaluate EACH comment (the core of this skill)

For every comment, verify the claim against the actual code and repo conventions before touching anything. Read the whole file, not just the diff hunk Codex saw, plus the types and the call sites. Then classify:

1. CORRECT + actionable: the finding is real and the fix improves the code. → fix it (step D).
2. FALSE POSITIVE / technically wrong: the claim doesn't hold. → do NOT change code; write a specific, evidence-based reply (cite the exact code/line/behavior that disproves it), then resolve.
3. Valid but out-of-scope / stylistic nit that conflicts with repo convention or the PR's intent → briefly decline with a reason, then resolve. Do not expand the PR's scope to satisfy a nit.

Hard rules:

• Never modify correct code just to silence Codex. Prefer a reasoned rebuttal.
• When uncertain whether a claim holds, investigate (read more code, run the type-checker / tests) rather than assume Codex is right. Default to skepticism.
• If a suggested change would break other call sites, alter public behavior, or contradict a verified repo convention, it is a category-2 rebuttal, not a fix.
• Never fabricate identifiers to satisfy a comment (e.g. a Linear/ticket prefix). If Codex asks for a ticket reference and none exists, say so; do not invent one.

Codex's common failure modes to watch for (default these to category 2):

• Diff-local reasoning: asserts a bug that the unchanged surrounding code already handles.
• "This can be null/undefined here" where the type or an earlier guard already rules it out.
• Invented race conditions or error paths with no actual trigger.
• Suggestions that compile-break or break other callers.
• Security/perf warnings with no exploit path or measurable cost.
• Restating library/framework semantics incorrectly.
• Style demands that contradict the repo's existing, consistent pattern.

D. Apply fixes: category 1 only

Make the minimal correct change. Re-run the local gate if the repo has one (typecheck/tests) before moving on.

E. Commit and push FIRST, before resolving anything

Order matters. A resolved thread is a claim that the fix is on the branch, so the push has to succeed before the claim is made: otherwise a failed commit or push leaves the PR unfixed with the finding marked resolved, and nobody looks at it again.

If step D changed code:

# Stage ONLY the files your fixes touched: never `git add -A`, which sweeps up
# unrelated work and untracked secrets sitting in the worktree.
git status --short                     # look before you stage
git add <path> [<path>...]             # the files named in the findings you fixed
git commit -m "address codex review feedback (codexloop iteration N)"
git push

Author the commit per the repo's norms (e.g. the user's identity; no AI attribution if that is the convention). Confirm the push actually landed before continuing:

git rev-parse HEAD
gh pr view <PR> --json headRefOid -q .headRefOid   # must match

If they differ, stop: the fix is not on the PR, so nothing may be resolved yet.

F. Reply to and resolve every addressed thread

Only now, with the fixes pushed, reply and resolve. Fetch unresolved threads, following pagination: a PR with more than 100 threads will otherwise look clean while unresolved findings sit on page two:

# Loop until hasNextPage is false, passing endCursor back in as $cursor.
CURSOR=null
while : ; do
  PAGE=$(gh api graphql -F cursor="$CURSOR" -f query='
  query($cursor: String) {
    repository(owner: "OWNER", name: "REPO") {
      pullRequest(number: PR_NUMBER) {
        reviewThreads(first: 100, after: $cursor) {
          pageInfo { hasNextPage endCursor }
          nodes { id isResolved comments(first: 1) { nodes { databaseId author { login } path body } } }
        }
      }
    }
  }')
  echo "$PAGE"          # collect nodes from every page before deciding the PR is clean
  PI='.data.repository.pullRequest.reviewThreads.pageInfo'
  [ "$(echo "$PAGE" | jq -r "$PI.hasNextPage")" = "true" ] || break
  CURSOR=$(echo "$PAGE" | jq -r "$PI.endCursor")
done

Reply on a thread's comment via gh api repos/{owner}/{repo}/pulls/<PR>/comments -f body="..." -F in_reply_to=<comment_id>, then resolve:

gh api graphql -f query='mutation { resolveReviewThread(input: {threadId: "THREAD_ID"}) { thread { isResolved } } }'

Resolve a thread only for comments authored by $BOT that you have fixed or rebutted: never blanket-resolve, and never resolve a human reviewer's thread.

Threads you are rebutting need no push, so they may be replied to and resolved regardless of whether step D changed code.

G. Re-review

Pushing re-triggers Codex when auto-review is on; otherwise post @codex review. Go back to A with the new head SHA. If step D changed nothing (all comments were rebutted), skip the push, ensure all threads are resolved, and exit.

3. Exit conditions

Stop when any is true:

• Zero unresolved $BOT comments remain, and every comment this round was fixed or rebutted+resolved. (There is no score to hit: this is "done".)
• Max iterations (5) reached: report what remains.
• Codex never responded after one retry: report the timeout honestly; do not claim success.

4. Report

Codexloop complete.
  PR:                 #<n>
  Bot login:          <resolved $BOT>
  Iterations:         N
  Comments fixed:     N   (genuinely-correct findings)
  Comments rebutted:  N   (false positives / nits, resolved with rationale)
  Remaining:          0

If it stopped at max iterations, list the remaining threads with your current assessment (fix-pending vs disputed) so a human can arbitrate.