Dependabot Audit

Dependabot Audit

This skill reports; it does not decide. It ends in a recommendation plus the evidence behind it, and stops. Do not merge, approve, close, comment on, rebase, or push to the PR. Print the merge command; do not run it.

disallowed-tools removes Edit, Write, and NotebookEdit from the pool while this skill is active, but Bash remains and can reach gh pr merge. That restraint is a contract, not a sandbox — honor it. There is no exception: Phase 8 hands its memory entry back rather than writing it, precisely because reaching for Bash to do what the withheld tools would have done makes the withholding theatre.

This audit executes the code it audits

The contract above governs what this skill writes. It says nothing about what the audited code does, and two phases run it:

• Phase 5 installs frozen and runs the PR's own test suite from the PR's tree. uv sync builds any sdist in the resolution, which runs setup.py or the project's build backend.
• Phase 4 runs the repo's gates at a version taken from the diff under audit.
• gate_diff.py passes its --run commands to a shell, and those commands are transcribed from the audited repo's CI config, which an actions bump legitimately modifies.

The worktree isolates the user's working tree from the audit. It does not isolate the machine from the PR. Nothing here is a sandbox; if you need one, it has to come from outside — a container, a throwaway VM, or a Landlock confinement — and this skill cannot verify that you have one.

The ordering is the mitigation available inside the skill, and it is worth being exact about what that buys. Phase 1 is a gate: if the diff reaches beyond the manifest and lockfile, or provenance fails, stop there. Do not continue into the phases that execute. A procedure whose thesis is "verify before you trust" must not run the artifact before it has finished deciding whether to trust it.

What the gate catches is a lockfile edited after it was written honestly — a hash, size, URL or yank status that disagrees with the registry — and a diff that reaches into source. It does not catch a malicious release. Phase 1 compares the lockfile against what the registry serves today, so when the attacker published the artifact, the record and the lockfile agree — and agreement is the entire test. A bump to a version whose maintainer account was compromised passes Phase 1 clean and arrives at Phase 5's install with the gate's blessing.

The one signal that speaks to it is PEP 740 build provenance: PUBLISHER CHANGED means the release being adopted was built somewhere the previous one was not. Coverage is partial and version-dependent, so where there is no attestation there is no signal. Read the ordering as what it is — it removes the cases it can see, and --no-execute is the answer for the rest.

--no-execute runs Phases 0–3 and 6–7 only. Every one of those is a network read: provenance, currency, changelogs, OSV, CI state. That is most of this procedure's value, and it is the right default for a PR you have no reason to trust yet. Use it when the user asks, and when Phase 0 classifies the PR as one the bots did not open. Say in the report which phases did not run.

That claim is a property of each phase's commands, not of its number, and it has already been false once: until 0.34.0 the uv.lock recipe for Phase 3 was uv run --with pip-audit …, which syncs the project — installing it editable and building any sdist in the resolution — so the mode that exists for a PR you do not trust ran that PR's build code. A phase in this set that gains a command has to be checked against this sentence, which is what tests/test_skill_prose.py now does mechanically.

Arguments

/dependabot-audit <PR> [--no-execute] [--comment], and the same words said in prose. The PR number is the only one that is required.

If no PR number arrived, ask which one before starting. Do not reach for the most recent bump, and do not audit whatever branch happens to be checked out. Both read as helpful, and both audit something the user did not ask about — the report that comes back is then about the wrong PR while looking exactly like a report about the right one.

• --no-execute — as above: Phases 0–3 and 6–7, and name the skipped phases in the report.
• --comment — produce the report and print it, then offer to post it. Posting is a separate action the user asks for explicitly; the flag requests the offer, not the post.
• Anything else is not a flag this procedure knows. Say so, rather than inferring what it might have meant.

Why this procedure exists

The failure modes that bite are not "is this package malicious" — they are a proposal that is already stale, a gap containing a fix no vulnerability database knows about, and a bump that changes a default rather than a behavior. All three are observed, not hypothetical. Phases 2 and 4 exist for them, and each carries the measurement it came from.

Phase 0 — Discover the repo (derive every run; never cache)

Never persist the answers to these. Required checks get added, CI jobs get renamed, and a cached profile silently audits a repo that no longer exists. Deriving costs one call each.

Derive with the script; mutate by hand. scripts/discover.py answers every derivable question and tags each answer derived / absent / underivable. It is read-only — no fetch, no worktree, no local git at all — so the two things Phase 0 changes in the user's repository stay visible in this file, where a plugin whose contract is "reports, never merges" should keep them.

D="${CLAUDE_PLUGIN_ROOT}/skills/dependabot-audit/scripts/discover.py"
REPO=$(gh repo view --json nameWithOwner --jq .nameWithOwner)
# OUTSIDE the repo, and the SAME directory on every later call — derived, not remembered
SCRATCH="${SCRATCH:-${TMPDIR:-/tmp}/dbaudit-${REPO/\//-}-<N>}"; mkdir -p "$SCRATCH"

python3 "$D" --repo "$REPO" --number <N>                              # the report
python3 "$D" --repo "$REPO" --number <N> --shell > "$SCRATCH/phase0.env"
. "$SCRATCH/phase0.env"

Source the outputs rather than transcribing them. Four of them are 40-character SHAs, and a wrong one is not detectable downstream: a truncated $HEAD_SHA matches no CI run and reads exactly like CI never ran, and a wrong $BASE_SHA gives a scope diff that is wrong rather than empty. What the file defines:

#   SCRIPTS=<abs path>     this plugin's own scripts/ — the one output not about
#                          the PR. Derived from discover.py's own location, so a
#                          reference can name a script; see below
#   DEFAULT=<branch>       the repo's default branch, derived
#   HEAD_SHA=<40 hex>      the commit under audit
#   BASE_REF=<40 hex>      GitHub's own base for the PR
#   BASE_SHA=<40 hex>      the merge base, from GitHub's compare endpoint
#   OWNER=<owner>          for Phase 6's GraphQL variables
#   NAME=<name>
#   CREATED_AT=<iso8601>   when the PR was opened — Phase 2's cooldown test
#   BRANCH_POINT=<ok|rewritten|suspect|underivable>
#   MAY_EXECUTE=<yes|no>   whether Phases 4 and 5 are authorised
#   HUMAN_COMMITS=<shas>   non-bot commits on the branch — a finding, not a gate
#   ECOSYSTEM=<uv.lock|github-actions|unsupported|unknown|underivable>
#   SCOPE_GATE=<clean|beyond|underivable>   Phase 1's gate, already decided

An underivable output is emitted commented-out, so it stays unset. That is deliberate: a later phase then fails loudly on an empty value instead of quietly on a plausible one, which is the distinction this whole phase exists to preserve.

Exit 2 means it could not run; exit 1 means it ran and found something. Never read one as the other. Exit 1 here does not stop the audit — it means the shape of the audit changes, and the report has to say how.

Read out of its output: $DEFAULT, $HEAD_SHA, $BASE_SHA, $PERMS, whether the merge base is the branch point, and whether Phases 4 and 5 may run at all. Take $BASE_SHA from it rather than from git merge-base, and the reason is the next section.

Then the part that changes state, which is yours:

# Fresh call: nothing survives one, so re-derive $SCRATCH and re-source Phase 0.
REPO=$(gh repo view --json nameWithOwner --jq .nameWithOwner); SCRATCH="${SCRATCH:-${TMPDIR:-/tmp}/dbaudit-${REPO/\//-}-<N>}"
. "$SCRATCH/phase0.env" || { echo "no handoff in $SCRATCH — re-run Phase 0" >&2; exit 2; }

git worktree prune          # a previous run's registrations, if $SCRATCH is gone
git fetch origin "pull/<N>/head:pr-<N>" "$DEFAULT"

# The fetch is the first thing that can disagree with the pin, so assert it here
# rather than trusting it. Exit 1: the block ran and found something, and what it
# found changes the shape of the audit.
FETCHED=$(git rev-parse "pr-<N>") || { echo "pr-<N> did not resolve after the fetch" >&2; exit 2; }
[ "$FETCHED" = "$HEAD_SHA" ] || {
  echo "the head moved: discover.py pinned $HEAD_SHA, the fetch resolved $FETCHED" >&2
  echo "re-run Phase 0 from the start — the pin, the base and the scope move together" >&2
  exit 1; }

git worktree add "$SCRATCH/pr-<N>" "pr-<N>"
git worktree add --detach "$SCRATCH/base-<N>" "$BASE_SHA"   # Phase 4 measures here

That assertion is not defensive clutter, and the interval it guards is seconds. A bot can rebase between discover.py reading the head and the fetch resolving it, and then every row of the report is about a commit that is no longer the PR. Measured on this plugin's own #99: discover.py ran at 23:08:39Z, Dependabot rebased at 23:08:57Z, and the run's next read returned a different head and a different base — $BASE_SHA moved with it, because the rebase moved the branch onto current main.

It was caught that time by accident, which is the argument for the assertion. git worktree add prints an abbreviated SHA, a reader noticed it disagreed with $HEAD_SHA, and the audit recovered. Nothing in the procedure compared the two; the guard below fires only when worktree add refuses, which it does not do here — the path was free and the add succeeded, at the wrong commit.

Re-run Phase 0 whole, never just the pin. The head, the base, the scope gate and the branch-point verdict are all derived from the same read, and a rebase moves more than one of them. Patching $HEAD_SHA alone leaves $BASE_SHA pointing at the pre-rebase base, which is the rewritten-base failure arriving by a different route.

prune first, and it is not defensive clutter. $SCRATCH lives under $TMPDIR, so a reboot or a tmp sweep between two audits of the same PR deletes the worktrees and leaves their registrations behind. Git then refuses the fetch — one command before any worktree add — with a message naming a directory that is not there:

fatal: refusing to fetch into branch 'refs/heads/pr-<N>' checked out at '<a path that no longer exists>'

That is this, not a permissions or ref problem, and the stale-worktree paragraph below does not reach it: that paragraph is keyed to git worktree add refusing, and Phase 0 never gets that far. prune is a no-op when state is clean, needs no path argument, and clears pr-<N> and base-<N> together.

Create the worktrees only where Phase 4 or Phase 5 will run. They are the two phases that need a tree; every other read here reaches the PR through git show at a ref. Five paths run neither, and the condition is the phases rather than any one of the five:

Every row's input exists before the decision: discover.py writes $ECOSYSTEM, $SCOPE_GATE and $MAY_EXECUTE one command earlier, and the flag is a word in the invocation. This rule used to name the ecosystem instead of the property, and two live runs deviated from it independently — each reasoning out that an uncovered ecosystem consumes no worktree either, and each writing the gap up rather than acting on it (#111). Naming one of five conditions needed four more exceptions; the property is the same in all five, so it is stated once.

The fetch stays on every path: git show needs the ref, Phase 6's merge simulation needs pr-<N>, and Phase 7 still has a branch to remove. Where the ecosystem is not yet known, the scope diff settles it and costs one command.

And the part no script can read for you — the bot's configuration, which decides whether a currency gap in Phase 2 is lag or a deliberate hold, and the repo's own verification commands. That stays here for the same reason the mutations do: pytest may be uv run pytest, tox, nox, or a make target, and only the workflow says so.

Read every one of them at a ref. The rest of Phase 0 is pinned to the PR and these were not — they ran in the user's checkout, so the answers came from whatever branch happened to be there:

# Fresh call: nothing survives one, so re-derive $SCRATCH and re-source Phase 0.
REPO=$(gh repo view --json nameWithOwner --jq .nameWithOwner); SCRATCH="${SCRATCH:-${TMPDIR:-/tmp}/dbaudit-${REPO/\//-}-<N>}"
. "$SCRATCH/phase0.env" || { echo "no handoff in $SCRATCH — re-run Phase 0" >&2; exit 2; }

# Derive the workflow list before reading it, at both refs. A repo can have
# several, and a guessed `ci.yml` either fails loudly on a repo that spells it
# `tests.yml` or — worse — succeeds and silently narrows the gate list to one.
git ls-tree --name-only "pr-<N>:.github/workflows/";    echo "pr list exit: $?"
git ls-tree --name-only "$BASE_SHA:.github/workflows/"; echo "base list exit: $?"

git show "pr-<N>:.github/dependabot.yml" 2>/dev/null || git show "pr-<N>:renovate.json"
git show "pr-<N>:.pre-commit-config.yaml"

# Then every name each list gave, at its own ref. Not one file: `<workflow>`
# stands for the whole list, and Phase 6 asks for the same list narrowed to what
# the diff touched.
git show "pr-<N>:.github/workflows/<workflow>"       # the gates Phase 5 reproduces
git show "$BASE_SHA:.github/workflows/<workflow>"    # the gates Phase 4 measures with

ls-tree on a directory that is not there exits 128 and says so, rather than printing nothing at exit 0 — so a repo with no workflows is distinguishable from a read that failed, which is the distinction the two lists exist to support.

Each phase's gates come from the tree it runs them in, and the two trees are not the same one: Phase 5 reproduces in $SCRATCH/pr-<N>, Phase 4 measures in $SCRATCH/base-<N> — or $SCRATCH/tip-<N>. One list run in both manufactures this phase's own worst outcome, an exit 2 that reads downstream as a gate failure. Observed auditing a merged bump: the checkout's ci.yml listed uv run actionlint, which arrived three PRs later, so in the PR's worktree it could not spawn — "could not run" reported as "ran and found something", by the procedure that is most careful about that distinction everywhere else.

A gate on only one side of the bump is itself a finding, and it is quiet in both directions. A gate since removed runs against a tree that never had it; a gate the PR adds never runs at all — and the second is the one that matters, because an actions or tooling bump can legitimately add its own. Diff the two lists and report the difference rather than picking a side.

If git worktree add refuses because the path already exists, a previous run left it there. Prove it still points at this PR's head before reusing it — a stale worktree silently audits the wrong commit and every result downstream is wrong. Compare against the pinned SHA rather than eyeballing a log line:

# Fresh call: nothing survives one, so re-derive $SCRATCH and re-source Phase 0.
REPO=$(gh repo view --json nameWithOwner --jq .nameWithOwner); SCRATCH="${SCRATCH:-${TMPDIR:-/tmp}/dbaudit-${REPO/\//-}-<N>}"
. "$SCRATCH/phase0.env" || { echo "no handoff in $SCRATCH — re-run Phase 0" >&2; exit 2; }

test "$(git -C "$SCRATCH/pr-<N>" rev-parse HEAD)" = "$HEAD_SHA"
git -C "$SCRATCH/pr-<N>" status --porcelain                       # must be empty

If either check fails, remove it and re-add — with python3 "$SCRIPTS/cleanup.py" --scratch "$SCRATCH" --pr <N> rather than a bare git worktree remove. The second check fails precisely when the tree is dirty, and a plain remove refuses on exactly that, so the remedy would fail on the state it was written for — and a previous run's Phase 5 residue would be discarded unread rather than saved.

Phase 0 outputs. Every later phase consumes these and nothing else:

$SCRIPTS is on that list because ${CLAUDE_PLUGIN_ROOT} reaches only this file. The token is substituted into SKILL.md's text at skill load — which is why the D= line above resolves, and why the variable itself measures empty in every shell (ROOT=[], marketplace install and --plugin-dir alike). A reference file is never injected: the model reads it off disk, so the token arrives at the shell intact and the path collapses to /skills/dependabot-audit/scripts/…. Two lines of references/uv-lock.md shipped that way from 0.15.0 until the first uv.lock replay ran them.

So the bootstrap happens once, here, and everything downstream derives from it. discover.py reports its own directory, and a path taken from the file that just ran cannot name a different copy than the one running — which is also the answer to the stale-cache hazard, where an invented export CLAUDE_PLUGIN_ROOT=…/0.22.1 pins a release into a cache that keeps every older version and then audits with it, silently and successfully.

$PERMS is not on that list, and the distinction is the point. It is read off discover.py's report here in Phase 0, where the execution gate and the actionability question both use it. It is not written to phase0.env: the shell handoff carries MAY_EXECUTE, which is the decision $PERMS was consulted to make. So a later phase that sources the handoff and reads $PERMS gets the empty string, and the table above is the list that crosses — anything else is Phase 0's own working state.

The reason is that $PERMS is a set of flags rather than a value: $PERMS.push is how the gate below addresses it, and there is no shell form of that. Reducing it to the one bit later phases actually branch on is what MAY_EXECUTE is.

And they branch on it, rather than being trusted to remember this table. The blocks in Phases 4 and 5 that run the audited repo's code open with

[ "${MAY_EXECUTE:-}" = yes ] || { echo "not authorised" >&2; exit 2; }

quoted here as an illustration — the runnable copies live in references/uv-lock.md § Phase 4 and § Phase 5, which is where they are read from.

Tested for yes, never against no, and the difference is the whole guard: a block whose handoff did not load sees the empty string, and != no is true of it. The one direction this must never fail in is open.

Diagnostics — emitted, deliberately unread: BASE_REF. It is Phase 0's own cross-check on the compare call and no later phase consumes it. Every other name the emitter writes is read by a block; that is the rule, and an exemption is a decision written down here rather than a name nobody happened to use.

If a later phase needs something not on this list, it belongs here rather than there. A phase that consumes what a later phase creates cannot be run in order, and that has now shipped twice — tests/test_skill_prose.py is what stops the third.

An output that could not be derived is not an output. Every row above has three states, not two: derived; absent, which is often a finding in its own right; and underivable, where the call failed or its precondition did not hold. discover.py tags each one and leaves an underivable output unset rather than emitting a plausible value.

The third state is the dangerous one because the ways it happens do not raise — they produce a real-looking answer that travels downstream as fact, and the report then says something false with full confidence. A phase handed an underivable input says so in its evidence row instead of proceeding on the value, and Phase 7 does not print a row whose input was never established. "Could not check" is a legitimate thing for this procedure to report. "Checked, found nothing" when you could not check is not.

Classify the PR before trusting it enough to run it. Dependabot and Renovate push their branches into the repository, so a dependency bump arriving from a fork did not come from the bot:

Any of the last three is a finding in its own right, and each changes the default: run --no-execute, report what the read-only phases found, and let the user decide whether to authorise Phases 4 and 5. Say plainly that those phases would run the PR's code.

The push row is the one easiest to argue away, so name the asymmetry it rests on. A bot PR on a repo you control proposes code you were going to run anyway, under gates you already trust — your own CI would run it too. A PR on a repo you cannot merge into proposes code you had no plan to run, and the comparison to CI stops holding: CI runs it in a fresh container with a scoped token, and this procedure runs it on your workstation with your credentials in the environment. $PERMS is already derived above, so this costs nothing to check.

$PERMS has the same three states, and the script gates on the call rather than the value. A failed repos/:owner/:repo writes its error body to stdout, so a capture succeeds and holds {"message": "Not Found", ...} — at which point push is not true and reads exactly like a pull-only account. The exit code is 1, which is what separates this from the branch-protection trap below where the same shape arrives at exit 0. Failing closed is right; the report saying "you lack push here" when the audit could not tell is not, and discover.py prints underivable rather than a permission set.

Pin the head SHA here and audit that one commit everywhere. The lockfile Phase 1 reads, the worktree Phase 5 reproduces in, and the CI run Phase 6 checks must all describe the same commit, or the report's evidence table asserts a coherence it does not have. Bots rebase, so this is not hypothetical: a rebase mid-audit leaves Phases 1–5 describing a commit that no longer exists while Phase 6 reports on the new one. Fetching once and working from pr-<N> makes them consistent by construction, and Phase 7 re-checks the SHA before you write.

Never audit the working tree. Whatever branch the user happens to have checked out is not the PR, and a lockfile read from it is indistinguishable from one read from the PR — it just quietly reports no changes. That holds for the repo's config as much as its content: the gate list and the bot config are read above at a ref for exactly this reason, and they were the last two reads here that were not.

SCRATCH has two requirements, and only one of them is about where it is. Never place it inside the repo under audit — it pollutes git status, and a gate that walks the tree (a linter, a formatter, a test collector) will descend into a full second copy of the project and report on it. And it must resolve to the same directory on every later call, because $SCRATCH/phase0.env is written by one call and sourced by another, and both worktrees are addressed the same way.

SCRATCH=${SCRATCH:-$(mktemp -d)} satisfied the first and failed the second, and that is why the line above derives the name instead. Measured against this harness, two separate calls: an export in the first is unset in the second, shell functions likewise, and each call is a new shell process. So ${SCRATCH:-…} found SCRATCH unset every time, mktemp -d returned a new directory, and the next call sourced a phase0.env that was not there — leaving $BASE_SHA, $HEAD_SHA and $DEFAULT silently empty downstream rather than erroring.

The working directory does survive, and is still not a way out. It carried between calls when it stayed inside the project; a call that ends outside has its cwd reset back. SCRATCH is required to be outside the repo, so it can never be reached that way. Nothing ambient crosses the boundary — only a path each call can recompute from what it already has, which is the repo and <N>.

Recomputable is not recomputed, and that distinction shipped as a defect. Deriving $SCRATCH made the handoff findable from a later call; it did not make any later call go and find it. So every block below that consumes a Phase 0 output opens with the same three lines, and they are not decoration:

# Fresh call: nothing survives one, so re-derive $SCRATCH and re-source Phase 0.
REPO=$(gh repo view --json nameWithOwner --jq .nameWithOwner); SCRATCH="${SCRATCH:-${TMPDIR:-/tmp}/dbaudit-${REPO/\//-}-<N>}"
. "$SCRATCH/phase0.env" || { echo "no handoff in $SCRATCH — re-run Phase 0" >&2; exit 2; }

Repeated rather than stated once, because a step that is merely implied is one that gets skipped — the same argument that put the gate list at a ref. The || is the load-bearing half: a . on a missing file returns 1 and keeps going, so without it the block runs on with every output empty, which is the state this whole phase exists to make impossible.

What that empties, measured, running each consuming block in a fresh call with nothing sourced: Phase 1's lockfile read, Phase 4, Phase 6 and Phase 7's cleanup all fail loudly — a Permission denied, two exit 2s, an exit 128 — and Phase 1's authorship gate passed silently, because for c in $BOT_COMMITS over an unset variable iterates zero times and handed the gate an empty file list. The gate is discover.py's from 0.29.0 and no longer reads that variable, but the class is unchanged and the handling belongs in the shell rather than only in the paragraph that describes it: an unset $SCOPE_GATE compares equal to nothing, so Phase 1's [ "$SCOPE_GATE" = clean ] fails closed into a stop rather than open into a pass.

git state is the exception and needs no reload. The pr-<N> ref Phase 0 fetches is in the repository, so git show "pr-<N>:…" works from any call. Only the shell handoff is lost.

A harness-provided SCRATCH still wins, and now for a reason: if one is exported into every call's environment it is stable by definition. The derived default is what applies when it is not.

Re-running the same audit reuses the directory rather than littering a new one, so Phase 7's cleanup is addressable from any call — but a stale worktree from an interrupted run is then in the way, which is a thing to remove rather than to work around.

Why the base comes from compare and not from git merge-base. Once a PR has landed its head is an ancestor of the default branch, so the merge base of the two is the head — and auditing a merged PR is a supported thing to do here: Phase 6 has a row for it, references/actions.md has a paragraph, and every replay this project's own gate asks for is one. Measured on cli/cli's merged bumps #14147, #14091, #13981 and #14049: git merge-base trunk pr-<N> returns the PR's own head for all four, so the scope diff is 0 files where GitHub reports 4, 2, 3 and 2. GitHub's compare endpoint returns the real branch point in both states, which is why the script uses it and why no phase runs a local merge base at all.

Prove the merge base is where the bot branched. A merge base always exists, and when the base branch has been rewritten under an open PR it is far too old — silently, with every later phase consuming it as fact. discover.py decides this and prints which case fired; what matters here is that the three cases are not interchangeable:

The substitutions, when rewritten fires:

• Phase 1 needs no substitution: its gate reads the bot's own commits, and a commit carries its own diff with no range to be wrong about. Where the authorship split is also underivable there is nothing safe to fall back to — the whole-diff range is the entire divergence — so the gate answers underivable rather than guessing.
• Phase 4 measures in $SCRATCH/tip-<N> rather than $SCRATCH/base-<N>, because the tree this PR would land on is the default branch's tip.

# Fresh call: nothing survives one, so re-derive $SCRATCH and re-source Phase 0.
REPO=$(gh repo view --json nameWithOwner --jq .nameWithOwner); SCRATCH="${SCRATCH:-${TMPDIR:-/tmp}/dbaudit-${REPO/\//-}-<N>}"
. "$SCRATCH/phase0.env" || { echo "no handoff in $SCRATCH — re-run Phase 0" >&2; exit 2; }

[ "$BRANCH_POINT" = rewritten ] || { echo "base not rewritten — no tip worktree" >&2; exit 0; }
git fetch origin "$DEFAULT"
git worktree add --detach "$SCRATCH/tip-<N>" "origin/$DEFAULT"

Say both substitutions in the report. "The base branch was rewritten under this PR" is a true and useful finding; "this bump reaches beyond the manifest and lockfile" is not, and they produce the same diff.

A two-parent head is not a moved base, and the script will not treat it as one. Measured on cli/cli #14049, whose head is "Merge branch 'trunk' into dependabot/…" by a maintainer above the bot's own commit: zero force-push events, and a correct two-file scope diff from the merge base. Read as a moved base it would substitute the pr-<N>^ diff — 20 files, 1,101 lines — and halt the audit on a bump that changes four workflow lines. There, pr-<N>^ is the branch tip, not the branch point.

Observed at the other end: a two-file Cargo.toml / Cargo.lock bump whose merge-base diff was 14 files and 3,682 deletions, appearing to delete the repo's entire vendored supply-chain/ tree. The base had been force-pushed eleven minutes after the PR opened, and the merge base fell back nineteen months.

gh pr view --json files is not a cross-check on any of this, which is why Phase 0 does not fetch it. GitHub computes the PR's file list from the merge base too, so on the force-pushed bump above it reported the same wrong 14 files. It agrees with the wrong answer rather than correcting it.

$PERMS decides two separate things, and conflating them gets both wrong. The tier that can merge is push; the tier that can read branch protection is admin. The common case — a maintainer with push but not admin — sits between them, and at pull only the verdict becomes a recommendation the reader cannot act on, so offer --comment text instead.

Do not try to read the required checks here at any tier. That question moved to Phase 6, which asks it per-PR in a form readable at pull. The two endpoints that look like they answer it both fail into a plausible value, in opposite directions: branches/<b>/protection needs admin and answers a bare 404 that is indistinguishable from an unprotected branch, while rules/branches/<b> reads at any tier and reports rulesets only, so classic protection returns [] and manufactures a false "nothing enforced". Both measured, in CONTRIBUTING.md.

Recalled project memory may already name landmines for this repo (Phase 8 writes them). Treat those as leads to check, not as facts — verify before repeating.

Phase 1 — Scope and provenance

Requires from Phase 0: $SCRATCH, $ECOSYSTEM, $SCOPE_GATE, $HUMAN_COMMITS, pr-<N>.

Phase 0 derived this gate; this phase acts on it. discover.py reads the files $BOT_COMMITS changed — never the branch's, because a maintainer can land the fixup the bump requires on the bot's own branch, and gated on the union that produces a Hold in language that reads exactly like a bump reaching into source. It answers in Phase 0's three states:

The count of files is not the invariant and never was: an action is pinned in every workflow that uses it, and a grouped bump moves several actions at once, so ordinary merged bumps touch two, three or four files. references/actions.md has the measurements, and the rule for reading the versions out of that diff rather than off the title.

underivable is not clean, and the asymmetry is the point. Every way this gate fails quietly arrives as no objection rather than as an error — an unset $BOT_COMMITS iterating zero times, an empty file list, a capped page hiding file 301 — and it is the gate that refuses Phases 4 and 5 a shell. Where the split is underivable the script falls back to the whole $BASE_SHA..pr-<N> diff and prints which source it used; read that line before quoting the verdict.

# Fresh call: nothing survives one, so re-derive $SCRATCH and re-source Phase 0.
REPO=$(gh repo view --json nameWithOwner --jq .nameWithOwner); SCRATCH="${SCRATCH:-${TMPDIR:-/tmp}/dbaudit-${REPO/\//-}-<N>}"
. "$SCRATCH/phase0.env" || { echo "no handoff in $SCRATCH — re-run Phase 0" >&2; exit 2; }

echo "ecosystem: $ECOSYSTEM"
# a maintainer's commit on the bot's branch: read it, report it, do not Hold on it
HUMANS=$(for c in $HUMAN_COMMITS; do git show --name-only --format= "$c" || exit 1; done) \
  || { echo "cannot read a commit in \$HUMAN_COMMITS" >&2; exit 2; }
printf '%s\n' "$HUMANS" | sort -u
# Last, so the half above still reports. Unset compares equal to nothing, so an
# empty $SCOPE_GATE stops here rather than reading as clean.
[ "$SCOPE_GATE" = clean ] \
  || { echo "scope $SCOPE_GATE — report it, and STOP before Phase 4" >&2; exit 1; }

A merge commit is in that list and normally prints nothing — its content arrived from the branch it merged. What it does print is what the merge itself changed, which is the one thing worth seeing there.

A provenance discrepancy stops the audit here too. Phases 4 and 5 execute the PR's code, and the point of running the cheap read-only checks first is that they can refuse to hand it a shell; continuing anyway spends the ordering for nothing. Stopping here is not a failed audit but a complete one that reached a verdict early — write the report with the phases that ran, and say which did not.

The method is per-ecosystem; the gate above is not. Each reference is sectioned by phase, so read the section for this one:

…(truncated)