Triage Pr Comments

Triage PR Comments

Go through review comments on the open PR for the current branch, verify each against the actual code (don't trust the reviewer blindly), classify them into explicit states, fix the valid ones, and post responses. Works for CodeRabbit, humans, or any reviewer.

For a read-only "where are we?" view before acting, use the pr-review-status skill — this skill is the active, destructive counterpart (edits code, posts replies, commits, pushes).

Prerequisites

• gh CLI authenticated (gh auth status — if token is expired, tell the user to run gh auth login and stop)
• Current branch has an open PR

Comment states

Every comment gets classified into exactly one state. The state drives what happens next:

One state per comment. If you're torn, pick the more conservative (e.g. needs-info over guessing invalid).

Workflow

1. Find the PR

gh pr list --head "$(git branch --show-current)" --json number,title,url,state

If nothing returned, tell the user and stop — don't guess at other branches.

2. Pull the comments

There are two comment surfaces on a PR. Pull both:

• Inline review comments (attached to specific lines):

  gh api repos/<owner>/<repo>/pulls/<num>/comments --paginate
  

• Issue-level comments (PR body / summary / general discussion):

  gh api repos/<owner>/<repo>/issues/<num>/comments --paginate
  

For CodeRabbit triage, inline comments are where the actionable findings live. Filter with jq:

gh api repos/<owner>/<repo>/pulls/<num>/comments --paginate \
  | jq 'map(select(.user.login | test("coderabbit"; "i"))) | .[] | {path, line, id, body}'

Save the output — you'll need the comment IDs later to reply.

3. Triage each comment

Critical: Do not just summarize the reviewer's claim. Verify it against the current code. Reviewers (especially automated ones) can be wrong, make claims based on stale code, or miss mitigations that already exist elsewhere.

For each comment:

1. Read the claim
2. Open the referenced file/line and check whether the claim is actually true right now
3. If the claim depends on other code paths (e.g. "the resolver only handles format X"), verify those paths too
4. Assign one of the five states above (valid-fix, partial, invalid, defer, needs-info)

For valid-fix items, assess priority:

• Runtime bug (will cause user-visible failures) → high
• Latent / future risk (no active path uses it today but will break when enabled) → low
• Defensive / hardening (silent misconfiguration prevention) → medium

3a. Grill on needs-info

Before moving on, resolve any needs-info classifications by asking the user targeted questions. Good reasons to land in needs-info:

• The comment is vague ("this could be cleaner")
• Two comments contradict each other
• The comment references behavior you can't confirm without domain knowledge
• The claim depends on intent ("should this throw or return null?") that only the user knows

Ask at most 2-3 questions per comment. Don't over-interview. If a single clarifying answer resolves it, re-classify and move on.

If the user's answer reveals the comment is actually valid or invalid after all, re-triage it — needs-info should empty out by the end of step 3a.

4. Report to the user

Write a concise triage summary — one section per comment with:

• File:line reference
• State (valid-fix / partial / invalid / defer) and why
• For valid-fix: priority (high / medium / low)
• Suggested Action

End with a priority order for the valid-fix items. Wait for the user to confirm which to address — don't auto-fix everything. For invalid and defer items, confirm the user wants to proceed with a reply (not post silently).

5. Implement the fixes — do not commit yet

Work through the approved valid-fix items. Do not commit or push automatically. The user reviews the diff before anything lands in history.

After the edits are in place:

1. Run git diff --stat and git diff (or show the relevant hunks) so the user can see exactly what changed
2. Summarize each fix briefly, one line per comment addressed
3. Wait for the user to confirm the changes look right

If the user wants edits, make them and re-show the diff. Don't move to step 6 until they explicitly approve.

6. Commit — only after user approval

Once approved, stage only the relevant files (don't git add -A) and commit with a message that lists each fix. Keep commits focused — don't bundle in unrelated cleanup.

Commit message template:

fix: address <reviewer> review on <branch> PR

- <one line per fix, explaining what and why>
- <...>

After committing, show git status and the short SHA. Do not push yet.

7. Push — only after the commit is approved

Ask before pushing ("want me to push?"). Only push when the user confirms. Use git push origin <current-branch> — do not force-push.

If git push is blocked by a hook, surface the block and ask the user whether to adjust the hook or push manually. Do not try to work around the block.

8. Draft reply comments — never auto-post

Hard rule: every reply requires explicit user approval before posting. No exceptions. This includes "obviously fine" valid-fix acknowledgments, replies the user dictated verbatim earlier in the conversation, and any follow-up replies after a code change. The user gets the final say on every word that lands on the PR.

"Earlier approval" is not transferable — if the user approved one reply and the code later changed, draft the new reply and re-confirm before posting it. Replies are public and can shape the reviewer relationship; treat them like sending an email under the user's name.

Draft a reply for every triaged comment, shaped by its state:

• valid-fix → Reference the fix commit short SHA. Describe what was done, name specific state/variable/function names so a future reader can find the change. Don't re-argue the claim.
• partial → Acknowledge the concern, explain where/why it's already mitigated (file:line if possible), no SHA needed.
• invalid → Respectfully disagree. Point to the code that shows why the claim doesn't apply. Keep it neutral — reviewers (especially bots) can be wrong, and that's fine. Flag to the user as "draft rebuttal" before posting.
• defer → Acknowledge the issue is real but out of scope. Link to a follow-up issue if one exists; otherwise say one will be filed.
• needs-info should not reach this step — resolve it in 3a first.

Show all drafts to the user in one message — full text of each, grouped by comment, so they can compare and edit holistically. Then stop and wait for explicit approval ("post them", "ship it", or per-reply edits). Do not interpret silence, agreement on a different topic, or general project approval as authorization to post.

9. Post replies — only after explicit approval

Post via the dedicated replies endpoint.

Gotcha: POST /pulls/<num>/comments with in_reply_to in the body does not work via gh api -f because -f stringifies and the API rejects the string. Use the dedicated replies endpoint instead:

gh api --method POST \
  repos/<owner>/<repo>/pulls/<num>/comments/<comment_id>/replies \
  -f body="$(cat <<'EOF'
<reply body here>
EOF
)" --jq '.id'

Post all approved replies in parallel (multiple Bash calls in one message). Report the reply IDs back to the user.

Notes

• If you're unsure whether a reply was approved, it wasn't — go back and ask.
• Needs gh, git, and jq; if one is missing, prompt the user with its install command from skill.deps.json.
• If the PR repo moved (GitHub redirects during push), the old owner/repo in git remote still works for gh api, but prefer the new location when constructing URLs.
• CodeRabbit comments include a lot of collapsed <details> blocks — the meaningful claim is usually in the first paragraph. Skim the rest only if you need the suggested diff.
• If a reviewer pushes back on a reply you posted, re-triage the comment from scratch. States aren't permanent — a valid-fix reply can still get disputed.