babysit-with-claude-review
Drive a pull request from "just opened" (or "ready for review") to merged +
clean worktree, in cycles around review feedback from Claude running on GitHub
(anthropics/claude-code-action).
Invocation: always /babysit-with-claude-review (hyphens, not colons). The
on-disk skill name is babysit-with-claude-review. Use this exact form in any
ScheduleWakeup prompt that re-enters this skill - a colon form is NOT a valid
command and will fail with "Unknown command".
Why this is not the Copilot loop
/babysit-with-github-copilot gates on a review event: after a re-request,
Copilot always submits a review summary, even "I reviewed your changes and
found no new comments". That event is the merge authorisation.
Claude on GitHub does not do this. Observed behaviour on a real PR
(OmnicodeSolutions/bora-turma#30):
| Copilot | Claude Code Review | |
|---|---|---|
| Clean review | posts an explicit "no new comments" review event | posts nothing at all |
| Findings | one review event with N inline comments | N review events, one per inline comment, every body empty |
| Trigger | --add-reviewer |
pull_request workflow event (auto), or @claude comment |
| Merge signal | review.submitted_at > last_push_at |
workflow run with headSha == <current head SHA> reaching completed |
Consequence: silence is ambiguous here. "Claude re-reviewed and found nothing" and "Claude has not started yet" look identical on the PR. Every copilot-style timestamp watermark on reviews or comments is therefore unusable as a gate. The workflow run is the only signal that distinguishes the two, and this skill gates on it.
Second consequence: since no review event ever says "clean", the PR is closed by an extra run rather than by a verdict. After fixing findings you push, let the review run again, and merge only if that run requested nothing new (Step 5). The trail left on the PR is one reply per finding plus a resolved thread (Step 4a) - never a summary comment.
Hard stops (read first - violations ship broken PRs)
Non-negotiable. If any would be violated, stop the turn and schedule the next cycle instead of merging.
- Never merge in the same agent turn as
git push. Push ends the turn. Schedule the next cycle (ScheduleWakeup, 300 s) and exit. - Never merge in the same agent turn as triggering a review. The run fires against the new commit; it needs time.
- Never merge without completing Step 3 at least once after the latest push. Replying to comment threads is not a substitute for waiting.
- Never treat "no CI checks reported" as permission to skip the wait.
- Never merge on silence. No comments from the reviewer bot is NOT a clean
review. Only a
completedreview run whoseheadShaequals the PR's currentheadRefOidauthorises merge. This is the master gate. - Never accept a run that did not review the current head commit. In auto
mode that means
run.headSha == <current headRefOid>exactly - a run on the previous commit reviewed code you have since changed, and a branch-name match is not enough. In mention moderun.headShais the default branch's tip, not the PR head (GitHub runsissue_commentworkflows against the default branch), so the SHA equality test can never pass; use the mention-mode matcher in Step 3 instead. Never relax the auto-mode test to a branch match to work around this. - Never count a run that skipped the review as a review - whether it
skipped at the run level or inside itself.
- Run-level
skipped. The mention workflow (.github/workflows/claude.yml, usually named "Claude Code") fires onissue_comment/pull_request_review/pull_request_review_commentand exitsskippedwhenever the trigger phrase is absent. Your own replies to review threads spawn a burst of these - on PR #30, tenskippedruns in six seconds. They are noise that looks exactly like review activity ingh run list. Gate on the review workflow only, matched by workflow file path (Step 0), and requireconclusion: success. - Self-skip inside a
successrun. The action refuses to review when the workflow file on the PR branch is not byte-identical to the version on the default branch, and it does so as a##[warning]in the step - so the run still reportscompleted+success, with zero comments. That is a merge-on-silence trap thatconclusioncannot see. Real case, run33479468831onmauriciovieira/skills:
Detect it in the log (Step 3). Do not usecompleted success fc1b82c ##[warning]Skipping action due to workflow validation: Workflow validation failed. The workflow file must exist and have identical content to the version on the repository's default branch.annotations_count- a genuinely clean run carries unrelated annotations (a real one hadannotations_count: 1, purely aNode.js 20 is deprecatedrunner warning), so counting annotations rejects good reviews and is not a test of anything. Grep the log for the marker instead.
- Run-level
- Never count a
failure,cancelled, ortimed_outrun as a review. The review did not happen. Re-trigger (Step 2) or surface to the user - a failed run posts no comments, which is indistinguishable from clean. - Never merge while the review run for the current head SHA is
queuedorin_progress.gh pr merge --delete-branchcancels it mid-scan and the comments it was about to post are lost. Observed run durations on a small repo: 51 s to 13 minutes. Budget accordingly; do not escalate early.
If you pushed fixes this turn: commit -> push -> ensure a review is triggered
(Step 2) -> brief status to user -> ScheduleWakeup -> stop. Do not call
gh pr merge until a later wake-up completes Step 3.
Preconditions (verify before starting)
- Current branch has commits ahead of
origin/main(or the repo's default). - Working tree clean (or only files you are about to commit yourself).
ghis authenticated for the repo.- Conventional Commits already used on the branch's history.
- The repo actually runs Claude on GitHub - verify in Step 0. If no Claude
review workflow exists, stop and tell the user to install it
(
/install-github-appin Claude Code, or add.github/workflows/claude-code-review.yml). Do not fall back to babysitting a PR with no reviewer. - A spec exists at
docs/superpowers/specs/YYYY-MM-DD-<topic>-design.mdif the repo follows that convention. Do not fabricate one - if absent and the change is non-trivial, flag it before opening the PR.
If any precondition fails, stop and report. Do not paper over.
Step 0 - Identify the review workflow and the reviewer login (once per PR)
Everything downstream keys off two values. Resolve them before the loop and persist them in loop state.
# Active workflows. You are looking for TWO different Claude workflows:
# - "Claude Code Review" / claude-code-review.yml -> the REVIEWER (gate on this)
# - "Claude Code" / claude.yml -> the @claude MENTION bot (noise)
gh api repos/<owner>/<repo>/actions/workflows \
--jq '.workflows[] | select(.state=="active") | [.name, .path, .id] | @tsv'
Pick the review workflow: the one whose path is claude-code-review.yml,
or - if named differently - the Claude workflow whose on: block contains
pull_request. Read it to be sure, and to learn the trigger style:
gh api repos/<owner>/<repo>/contents/.github/workflows/<file>.yml \
--jq '.content' | base64 -d | sed -n '1,40p'
on: pull_request-> auto mode. Every push to the PR starts a review run by itself. Step 2 is a no-op.on: issue_comment+trigger_phrase-> mention mode. Nothing runs until you post the trigger phrase. Step 2 is mandatory after every push.
A repo can have both. If the review workflow is auto, prefer it and treat mention mode as a manual re-trigger for when a run did not fire.
If this PR touches the review workflow file itself, it cannot be reviewed.
The action requires the file to be byte-identical to the version on the default
branch and self-skips otherwise - while still reporting success (Hard Stop
#7). Land the workflow change on the default branch first, then rebase or merge
it into the PR so the two match, and only then expect a review. Check with:
git show origin/main:.github/workflows/<file>.yml | git hash-object --stdin
git hash-object .github/workflows/<file>.yml
Equal hashes, or no review.
The same trap fires without your PR touching anything. Byte-identity is checked against the default branch as it is now, so the moment anyone merges a change to the review workflow, every open PR whose branch predates it stops being reviewed - silently, with green checks. Nobody edited those PRs; the ground moved under them.
Measured on OmnicodeSolutions/platform-infrastructure, where PR #44 changed
the workflow on main:
main blob 8e38de32
branches of #39,#43,#45 blob 8d40b02e
Three open PRs, all diverging, all self-skipping. #43 and #39 show
claude-review green with zero reviews and zero comments - indistinguishable
from a clean review unless you read the log. On
AxiomGovernance/platform#31 the same shape ran further: 62 files, 7513
insertions, five green runs across five SHAs, every one self-skipped, and
the PR was about to be used in a demo as reviewed code.
Step 3 already catches this - do not add a per-cycle hash check. A
self-skipped run is success with the marker in its log, and Step 3 greps for
that marker on every cycle regardless of why the skip happened. Whether the PR
diverged because it edited the file or because the default branch moved
underneath it, the gate closes the same way. The hash check above is for
diagnosing why once the gate has already closed, not for polling.
What this section is for is the other direction: a green check on a PR nobody
is babysitting. That is where these go unnoticed - #43 and #39 sat green
and unreviewed, and #31 was five runs deep. If a workflow change lands on the
default branch, every open PR older than it is in that state until someone
looks.
The fix is per PR: merge the default branch into the branch (not cherry-pick
the file - see the note under Step 3 on commit_id and merge bases), then push
to trigger a review that will actually run.
Reviewer login. Default is claude[bot], but the action posts under a
different account when the workflow overrides github_token (commonly
github-actions[bot]). Detect rather than assume:
gh api repos/<owner>/<repo>/pulls/<N>/comments --paginate \
--jq '[.[].user.login] | unique'
Persist as reviewer_login. Match with startswith("claude") OR the exact
detected string; never hardcode across repos.
Loop state to track between cycles
Persist across wake-ups (in the wakeup prompt or session notes):
| Field | Meaning |
|---|---|
<N> |
PR number |
review_workflow |
Workflow file, e.g. claude-code-review.yml (from Step 0) |
trigger_mode |
auto or mention (from Step 0) |
reviewer_login |
e.g. claude[bot] (from Step 0) |
head_sha |
PR's current headRefOid - the thing the gate matches on |
review_run_id |
Run id of the review for head_sha, once found |
trigger_posted_at |
Mention mode only: created_at of the @claude comment you last posted |
last_push_at |
ISO timestamp of the most recent fix push (empty if none) |
awaiting_rereview |
true after a fix push until Step 3 completes once post-push |
head_sha is the watermark, not a timestamp. Every push changes it, which
invalidates the previous run and closes the gate automatically - no clock
comparison, no propagation race.
Reset awaiting_rereview to false only on a wake-up (never on the push turn)
where Step 3 found a completed + success review run for the current
head_sha, no unresolved threads, and a clean sticky. It is belt-and-braces on
top of the SHA watermark: the watermark closes the gate, this flag stops you
merging on the same turn you pushed.
Step 1 - Open the PR (if not already open)
- Lint + test locally first. Repo conventions take precedence
(
make lint && make test). Do not push if lint or unit tests fail. - Rebase onto the upstream default branch:
git fetch origin --prune && git rebase origin/main - Push:
git push -u origin <branch>. - Compose the PR body from the repo's template at
.github/WORKFLOW_TEMPLATES/pull_request.md(orPULL_REQUEST_TEMPLATE.md). Fill it in fully - no placeholder bullets, no "TBD". Sections expected: Summary, Context, What changed, Test plan, Risk & rollback, Checklist. - Open via
gh pr create --title "<conventional title>" --body-file <path>. Pass the body via--body-file, never inline--body "$(cat ...)". - Capture the PR number
<N>; the rest of the loop hangs off it.
Step 2 - Trigger the review
Auto mode: nothing to do. The pull_request event already queued a run for
the new head SHA. Confirm in Step 3 that a run exists for that SHA; if none
appeared after two cycles, fall back to the mention form below.
Mention mode: post the trigger phrase as a PR comment.
gh pr comment <N> -R <owner>/<repo> \
--body "@claude review the latest commit on this PR"
# Record the trigger watermark - Step 3's mention-mode matcher needs both.
trigger_posted_at=$(gh api repos/<owner>/<repo>/issues/<N>/comments --paginate \
| jq -s -r 'add | sort_by(.created_at) | last | .created_at')
head_sha=$(gh pr view <N> -R <owner>/<repo> --json headRefOid --jq '.headRefOid')
Notes:
- A new
@claudecomment is the only way to re-trigger. Replying inside an existingclaude[bot]review thread does not re-trigger a review - it only spawns askippedrun of the mention workflow (Hard Stop #7). - Comment-triggered runs report
headBranch: mainandheadSha= the default branch's tip, not your PR branch or its head commit. Filter them by workflow +createdAt >= trigger_posted_at, and assert separately thatheadRefOidhas not moved since the trigger. Never filter by branch, and never expectheadShato equal the PR head in this mode. - Do not request other human reviewers unless the user told you to.
Step 3 - The wait cycle (mandatory gate)
You must sleep before deciding to merge. Use ScheduleWakeup with
delaySeconds: 300 and a prompt that re-enters this skill. Do not poll in a
tight loop; do not skip because "CI is empty" or "comments look addressed".
On wake-up, run all of the following before deciding the next move:
head_sha=$(gh pr view <N> -R <owner>/<repo> --json headRefOid --jq '.headRefOid')
gh pr checks <N> -R <owner>/<repo> # CI status
# The gate. Match on the REVIEW workflow file (--workflow takes the file name,
# which is stable across workflow renames), then narrow by trigger_mode.
runs=$(gh run list -R <owner>/<repo> --workflow "<review_workflow>" --limit 20 \
--json databaseId,headSha,status,conclusion,createdAt,updatedAt)
case "<trigger_mode>" in
auto)
# The run reviewed the PR head itself, so match the SHA exactly.
run=$(printf '%s' "$runs" | jq --arg sha "$head_sha" \
'map(select(.headSha == $sha)) | sort_by(.createdAt) | last') ;;
mention)
# issue_comment workflows run against the DEFAULT branch, so headSha is
# main's tip and can NEVER equal $head_sha - matching on it deadlocks the
# loop forever. Match on "started after the trigger comment I posted", and
# separately assert the PR head has not moved since the trigger; if it has,
# the run reviewed superseded code.
if [ "$head_sha" != "<head_sha when trigger was posted>" ]; then
echo "head moved since the trigger - gate closed, re-trigger (Step 2)"
run=null
else
run=$(printf '%s' "$runs" | jq --arg t "<trigger_posted_at>" \
'map(select((.createdAt|fromdateiso8601? // 0) >= ($t|fromdateiso8601? // 0)))
| sort_by(.createdAt) | last')
fi ;;
esac
run_id=$(printf '%s' "$run" | jq -r '.databaseId? // empty')
run_status=$(printf '%s' "$run" | jq -r '.status? // "none"')
run_concl=$(printf '%s' "$run" | jq -r '.conclusion? // "none"')
run_start=$(printf '%s' "$run" | jq -r '.createdAt? // empty')
printf 'gate: sha=%s run=%s status=%s conclusion=%s\n' \
"$head_sha" "${run_id:-none}" "$run_status" "$run_concl"
Then, only if run_status is completed and run_concl is success,
confirm the run did not skip the review inside itself (Hard Stop #7). A
success run that self-skipped posts zero comments and is indistinguishable
from a clean one on conclusion alone:
Fetch the log and grep it as two separate operations, and fail closed.
Piping gh run view --log straight into grep collapses "the log says no
skip" and "the log never arrived" into the same empty output, and the grep then
reports clean - reopening the exact merge-on-silence hole this check exists to
plug. Logs go missing for reasons unrelated to the code: GitHub expires them
after 90 days, and rate limits or network failures hit at any time.
log=$(mktemp)
if ! gh run view "$run_id" -R <owner>/<repo> --log > "$log"; then
echo "could not fetch log for run $run_id - gate CLOSED (cannot verify)"
# Retry next cycle; escalate after 3 consecutive fetch failures (see below).
elif grep -qiE 'workflow validation failed|skipping action due to workflow' "$log"; then
echo "run $run_id self-skipped - NOT a review, gate CLOSED"
# Usual cause: this PR edits the review workflow file, so it no longer
# matches the default branch. See Step 0. Do NOT merge.
elif grep -q '"is_error": *true' "$log"; then
echo "run $run_id errored inside the action - NOT a review, gate CLOSED"
else
# The action's result JSON. Echo these every cycle, always.
grep -oE '"(subtype|is_error|num_turns|permission_denials_count)": *[^,]*' "$log" \
| tail -4
denials=$(grep -oE 'permission_denials_count"?[: ]+[0-9]+' "$log" \
| grep -oE '[0-9]+$' | sort -rn | head -1)
turns=$(grep -oE '"num_turns": *[0-9]+' "$log" \
| grep -oE '[0-9]+$' | sort -rn | head -1)
if [ -z "$denials" ] || [ -z "$turns" ]; then
echo "run $run_id: result fields missing from log - gate CLOSED (cannot verify)"
# Do NOT default these to a passing value, and do NOT retry: escalate now.
# Missing means the log format changed and this check is no longer testing
# anything - refetching the same run yields the same gap. Surface to the
# user and fix the parser; never merge on it.
elif [ "$denials" -gt 0 ] && [ "$turns" -le 2 ]; then
echo "run $run_id: $denials denials in only $turns turns - blocked, gate CLOSED"
# Permissions problem, not something a retry fixes. Surface to the user.
elif [ "$denials" -gt 0 ]; then
echo "run $run_id: $denials denials but $turns turns - reviewed; FLAG to the user"
else
echo "run $run_id genuinely reviewed ($(wc -l < "$log") log lines)"
fi
fi
On permission_denials_count: denials alone are not the signal - denials
plus a run that barely did anything are. A run can finish green having been
blocked from doing its job: a documented case reviewed nothing in 2m49s with
permission_denials_count: 16. But a nonzero count on its own does not mean
that. Run 33479917884 on mauriciovieira/skills had
permission_denials_count: 1 alongside subtype: success, is_error: false
and num_turns: 4, and was a genuine clean review - gating on > 0 alone
would have wedged that PR for nothing.
So the gate closes on denials > 0 AND num_turns <= 2: a review stopped
from working shows up as both at once. Anything else with denials is treated as
reviewed but flagged to the user, never swallowed.
The
num_turns <= 2cutoff is provisional. It sits below the confirmed-good runs on this repo and above a review that did nothing, but it is fitted to a handful of observations, not calibrated. If a real review ever trips it, raise the evidence rather than deleting the check - and if a blocked run ever slips past it, tighten the cutoff. Record what you saw either way.
Observed baseline. Every run below was a genuine review on
mauriciovieira/skills, with the narrow allowlist described under
"What denials actually measure":
num_turns |
2 | 4 | 5 | 9 | 14 | 17 | 23 |
|---|---|---|---|---|---|---|---|
permission_denials_count |
0 | 1 | 1 | 1 | 3 | 4 | 14 |
What denials measure is still unknown, and one theory is already dead.
The count rises with num_turns, which fits per-attempt tool denials during
exploration and does not fit a fixed GitHub-token permission wall - that
would give a roughly constant count driven by how often the reviewer tries to
comment. That much still holds, and it is why the permissions: block was left
at read rather than widened: raising token permissions on the wall theory
would grant real access for an unverified benefit.
The specific theory that the denied tools were Read, Grep and Glob did
not survive its first test, but read the test's limits before treating it
as settled. The workflow allowed exactly one tool, so those looked like the
obvious candidates. Widening --allowedTools to include them changed nothing:
num_turns |
denials | |
|---|---|---|
before (run 33479917884) |
4 | 1 |
after (run 34371914614) |
4 | 1 |
What that shows, and what it does not. Both runs were on small markdown-only PRs, where the diff is the whole content: no file worth opening, no call to follow, no surrounding context. That is the one case where those three tools have nothing to do even when allowed. So the result is equally consistent with "widening changed nothing" and with "the reviewer never attempted them here" - it does not distinguish the two.
What it does establish: a denial appears at a low turn count regardless of that
widening, so at least one refusal comes from something else. Bash and any
other tool outside the allowlist remain candidates. The run log carries only
the count, never the denied tool name, so naming it needs a source the log does
not provide.
A real test needs a PR large enough that a reviewer would actually reach for those tools. Until then, do not record this as closed in either direction.
Two things follow for reading the table:
- A low turn count alone proves nothing. The
2 / 0row is a PR that vendored one file verbatim - little to review, so few turns and no denials. Compare like with like before concluding anything from a drop. - Do not re-run the widening experiment on a small PR. It has been done and it cannot discriminate. Any repeat has to be on a PR big enough that the reviewer would reach for a file it cannot see from the diff.
A separate finding, still open. Review depth does not scale with PR size.
MarcaCerta/marcacerta#23 - 84 files, 9027 insertions - was reviewed in 7
turns and 22 seconds and produced zero findings, including a residue its own
author had documented. A one-file markdown PR on this repo took 23 turns. If
the reviewer is reading shallowly, the allowlist is one suspect and
fetch-depth: 1 in the checkout step is another: with no history the reviewer
cannot compare against the base or follow how a file got that way. Neither has
been tested.
is_error: true closes the gate on its own; that one is unambiguous and needs
no threshold.
This rule has a test: test/verdict.sh, run with bash test/verdict.sh. It
covers the four verdicts against trimmed real run logs plus synthetic blocked
and renamed-field cases, and asserts the branch conditions still appear in this
file - so editing one without the other fails. Change the rule above and the
test together.
Never default a missing field to a passing value. These fields come from
the action's result JSON, which is not a stable contract - if num_turns is
renamed upstream, the grep stops matching and a ${turns:-99} style default
would silently turn this check into a constant pass, green forever with nothing
behind it. Absent fields close the gate, same as an unfetchable log: the check
is not reporting "clean", it is reporting that it can no longer test anything.
This fails loudly across every PR at once when the format changes, which is the
point - a gate that wedges the repo gets fixed in minutes, while one that
degrades to a warning disappears into the noise.
Missing fields escalate immediately; a failed log fetch retries 3 times first. The two fail-closed paths differ on purpose. A fetch failure is often transient - a rate limit or a network blip clears on the next cycle - so retrying can recover it. Absent fields cannot be recovered by retrying: the log downloaded fine, and parsing the same bytes again produces the same gap. The accepted cost is a truncated log, where the fields are missing for a transient reason and one refetch would have worked. That is rare enough to pay for by escalating early; the common cause is an upstream rename, which no number of retries fixes.
An unreadable log is an unverifiable run, and unverifiable is never clean. This can wedge a PR for a reason that has nothing to do with its code - that is the intended trade: every ambiguous state in this skill resolves to "wait", and escalating to a human beats merging something no one checked.
Why 3 fetch failures here, but ~6 cycles in Step 4 #5. The two waits are not the same measurement, and the numbers should not be unified. Step 4 #5 waits on a run still executing - progress is happening, and runs have been observed taking 13 minutes, so escalating at 3 cycles would interrupt normal work. This wait is on a finished run whose log will not load: that is unavailability, not slowness. Rate limits and network blips clear in a cycle or two; a log expired past GitHub's 90-day retention never loads at all, and retrying it six times changes nothing. Escalate sooner because waiting longer buys no new information. Do not "fix" this to 6 for consistency.
Only once the run is confirmed genuinely reviewed, read what it posted. Two surfaces, both needed:
# Inline findings. Claude posts one review event per comment, all with empty
# bodies, so /reviews tells you nothing. Ask for UNRESOLVED THREADS instead of
# filtering raw comments: a thread you answered and resolved in Step 4a is
# addressed by definition, and a new finding always arrives as a new
# unresolved thread. One query gives both the state and the content.
gh api graphql -F owner=<owner> -F repo=<repo> -F number=<N> -f query='
query($owner: String!, $repo: String!, $number: Int!) {
repository(owner: $owner, name: $repo) {
pullRequest(number: $number) {
reviewThreads(first: 100) {
nodes {
id isResolved isOutdated
comments(first: 1) {
nodes { databaseId author { login } path line createdAt body }
}
}
}
}
}
}' \
| jq --arg login "<reviewer_login>" \
'[.data.repository.pullRequest.reviewThreads.nodes[]
| select(.isResolved == false)
| .comments.nodes[0]
| select(.author.login | startswith("claude"))]
| {count: length, findings: .}'
# Sticky summary comment, when the workflow sets use_sticky_comment. It is
# UPDATED IN PLACE, so filter on updated_at, never created_at.
#
# This surface is a VERDICT, not a finding. Its existence decides nothing -
# read the body. See "Reading the sticky verdict" below.
gh api repos/<owner>/<repo>/issues/<N>/comments --paginate \
| jq -s --arg login "<reviewer_login>" --arg start "$run_start" \
'add
| map(select(.user.login == $login
and (.updated_at|fromdateiso8601? // 0) >= ($start|fromdateiso8601? // 0)))
| {count: length, bodies: [.[].body]}'
Never filter inline comments by
commit_id. GitHub rewrites a review comment'scommit_idto the PR's new head every time you push, so a comment from round 1 that you already fixed, replied to, and resolved reappears as if it were written against the current commit. Itsoriginal_commit_idkeeps the sha it was really about. Observed onmauriciovieira/skills#15: comment3916342744,created_at2026-09-02,original_commit_id: 515d28f,commit_id: a501711,isResolved: true- the round-2 review posted nothing at all, yet acommit_id == head_shafilter reported one finding. A loop using that filter can never converge: every push resurrects every comment it has already answered. Gate on thread resolution, which is the actual state of "addressed".
Reading the sticky verdict
The two surfaces mean different things, and counting both the same way is wrong. Inline comments are findings - one per issue, each on the line it is about, and any of them blocks the merge. The sticky comment is a summary of the whole review, so a clean review produces one too:
## Code review
No issues found. Checked for bugs and CLAUDE.md compliance.
That is count: 1 on the sticky surface and means the opposite of a finding.
Treating it as unaddressed feedback wedges a PR the reviewer just approved.
So: gate on the inline count, and on the sticky body.
sticky=$(gh api repos/<owner>/<repo>/issues/<N>/comments --paginate \
| jq -s -r --arg login "<reviewer_login>" --arg start "$run_start" \
'add
| map(select(.user.login == $login
and (.updated_at|fromdateiso8601? // 0) >= ($start|fromdateiso8601? // 0)))
| last | .body // ""')
# EVERY substantive line must carry the clean verdict - not just some line
# somewhere. Drop headings and blanks first, then require that no remaining
# line lacks the phrase. A bare substring search over the whole body would
# call this CLEAN:
# ## Code review
# **Security:** no issues found.
# **Correctness:** 2 bugs, see inline comments.
lines=$(printf '%s\n' "$sticky" | grep -vE '^[[:space:]]*(#+.*)?[[:space:]]*$')
if [ -z "$sticky" ]; then
echo "no sticky verdict this run - inline comments decide"
elif [ -n "$lines" ] \
&& ! printf '%s\n' "$lines" | grep -qivE 'no issues found|found no issues'; then
echo "sticky verdict: clean"
else
echo "sticky verdict NOT recognised as clean - gate CLOSED, read it yourself:"
printf '%s\n' "$sticky"
fi
The clean-verdict pattern is provisional. It matches the only real clean sticky observed (
mauriciovieira/skills#14). An unrecognised body closes the gate rather than passing, so a reworded verdict costs you a manual read and a pattern update - never a silent merge. Widen it from observed bodies, and never invert it into "assume clean unless it looks bad": that hands every future wording a free pass, which is this skill's core failure mode.Match the whole body, never a substring. The check above is "no line fails the pattern", not "some line matches it". A sectioned verdict that clears one dimension and flags another contains the clean phrase while carrying findings, and a bare
grep -qiEwould open the gate on it - the same free pass the paragraph above forbids, arriving through a recognised body instead of an unrecognised one.
Interpret:
- No run for
head_sha-> the review has not been triggered for this commit. Gate closed. Auto mode: wait one cycle. Mention mode: go to Step 2. - Run
queued/in_progress-> gate closed, wait (Hard Stop #9). - Run
completed, conclusion notsuccess-> the review did not happen. Gate closed (Hard Stop #8). Inspect withgh run view $run_id -R <owner>/<repo> --log-failed, then re-trigger or surface to the user. - Run
completed+success, but the log could not be fetched -> the run is unverifiable. Gate closed. Retry next cycle; after 3 consecutive failures surface to the user. Never merge a run you could not check. - Run
completed+success, but the log grep matched -> the action skipped itself; no review happened. Gate closed (Hard Stop #7). Fix the cause- almost always this PR editing the review workflow file - and re-run.
- Run
completed+success, log grep clean, zero UNRESOLVED threads, and the sticky body absent or recognised as clean -> genuinely clean review. Merge-eligible. The successful, non-self-skipped run on the matching SHA is what makes that meaningful. - Run
completed+success, sticky body present but not recognised as clean -> gate closed. Read it; it may carry findings the inline surface does not. Do not merge on an unread verdict. - Run
completed+success, >= 1 unresolved thread -> unaddressed until fixed and pushed (or replied "won't fix" with a reason). Go to Step 4 #3. A thread you already answered and resolved does not count - that is why the query filters onisResolved, not on a comment count.
Step 4 - Decide
Decision tree, in order. If the answer sends you to Step 3, do not merge this turn.
- CI failing -> investigate, reproduce locally, push a fix. Set
last_push_at,awaiting_rereview=true, trigger review (Step 2). Stop turn -> Step 3. Never merge with red CI. - CI still pending after 5 minutes -> wait one more cycle, then escalate to the user if still pending; usually a stuck runner.
- Review comments on the current head SHA -> address every comment with
substance. Skip nothing without justification (a "won't fix" gets a one-line
reply with the reason). Run lint + tests again. Commit + push. Set
last_push_at,awaiting_rereview=true, refreshhead_sha. For every addressed comment, complete Step 4a (reply + resolve + echo) before triggering the re-review. Then Step 2. Stop turn -> Step 3. Do NOT merge. awaiting_rereviewis true and this is the push turn -> waited time has not elapsed. Stop, ScheduleWakeup.- No review run for the current
head_sha, or it isqueued/in_progress-> wait one cycle (ScheduleWakeup 300 s). Do NOT merge (Hard Stops #5, #9). Runs have been observed taking 13 minutes; do not escalate before6 consecutive empty cycles (30 min), then surface to the user and let them decide. Never auto-merge on silence. - Review run for
head_shacompleted but notsuccess-> Hard Stop #8. Re-trigger or surface. Do NOT merge. - CI green (or no checks) AND a review run exists for the current
head_shaAND it iscompletedwith conclusionsuccessAND both comment surfaces return zero for that run -> merge. Go to Step 5.
Branch #7 is the only path to merge.
Step 4a - Reply, resolve, and echo every addressed comment (mandatory)
Exactly one reply per finding, on that finding's own thread, then resolve it. Never aggregate several findings into one comment, and never substitute a single summary comment on the PR for the per-finding replies - the reply has to sit on the thread it answers, or the reviewer cannot tell which finding it closed.
After pushing the fix commit(s), for each comment you addressed (whether applied or "won't fix"):
- Compose the reply text. One concise sentence: what changed, where, why.
- Applied:
"Applied in <sha> - <one-line summary> (file:line)." - Won't fix:
"Not taking this - <reason>. (Tracked: <link/issue> if any.)"
- Applied:
- Echo the reply to the agent terminal BEFORE posting. Print exactly what
you are about to send, prefixed with the comment's
path:lineand the reviewer login. The user must be able to read the verbatim reply text in the transcript without opening GitHub. - Post the reply on the comment thread:
Each of these spawns agh api -X POST \ repos/<owner>/<repo>/pulls/<N>/comments/<comment_id>/replies \ -f body="<reply text>"skippedmention-workflow run. Expected; ignore them (Hard Stop #7). - Resolve the review thread. Get the thread id via GraphQL by matching the
comment's
databaseId:
Then resolve:gh api graphql -F owner=<owner> -F repo=<repo> -F number=<N> -f query=' query($owner: String!, $repo: String!, $number: Int!) { repository(owner: $owner, name: $repo) { pullRequest(number: $number) { reviewThreads(first: 100) { nodes { id isResolved comments(first: 1) { nodes { databaseId } } } } } } }'gh api graphql -f threadId="<thread_id>" -f query=' mutation($threadId: ID!) { resolveReviewThread(input: {threadId: $threadId}) { thread { id isResolved } } }' - Verify
isResolved: truein the mutation response.
A comment is not "addressed" until reply + resolve + terminal echo all complete. Triggering a re-review with unresolved threads is a Step 4 #3 violation.
Step 5 - The extra-run gate (post-fix only)
Applies whenever you pushed a fix for review feedback, i.e. awaiting_rereview
is true. You may never merge on the same run that produced the findings.
Fixing the comments and merging is one round short: the fix itself is
unreviewed code.
The merge condition for the PR is therefore an extra review run, started after the fix push, that requested nothing new:
- The run exists in GitHub Actions for the current
head_sha(Step 3's matcher, pertrigger_mode). - It reached
completedwith conclusionsuccess- notskipped, notfailure(Hard Stops #7, #8). - Both comment surfaces return zero new comments from
reviewer_loginfor that run.
Only all three together authorise the merge. Two of three is a wait.
Throughout this skill, "gate CLOSED" means do NOT merge - the gate is a barrier, and a closed one blocks. Never read it as "close the PR", which is the opposite outcome.
Echo the gate to the agent terminal before merging, so the transcript shows which run authorised it:
gate: run <run_id> (<review_workflow>) on <head_sha> -> completed/success
at <run_updated_at>, 0 new comments from <reviewer_login>. CI: <status>.
Do not post this as a PR comment. The PR's audit trail is one reply per finding plus a resolved thread (Step 4a) - a summary comment on top of that is noise, and on a PR with no findings there is nothing to report.
Then Step 6.
What an open gate does not mean
Every hard stop above defends one sentence: the reviewer examined this commit and asked for nothing. That is all an open gate asserts. Three things it does not:
- It does not say the code works. Review reads a diff. It cannot run the
thing. A real case from a sibling project: a container that would not boot
because corepack cached the pnpm tarball in root's home during the build
while the runtime ran as
node, so every boot re-downloaded pnpm from the registry and hung on a prompt. That bug does not exist in the diff - it exists in the interaction between build and runtime. No allowlist, nofetch-depth, no reviewer of any depth catches it. Running the image catches it in one try, and the check would have been green with it inside. - It does not say the review was thorough. Depth does not track PR size.
MarcaCerta/marcacerta#23, 84 files and 9027 insertions, was reviewed in 7 turns and 22 seconds with zero findings, missing a residue its own author had documented. An 11-file infrastructure PR on another repo took 9 turns and found a real cross-file bug. Size predicts nothing. - It does not transfer judgement. The gate is a floor, not a verdict. It stops the specific failure of merging on silence; it does not
…(truncated)