Github Org Audit Runner

github-org-audit-runner

A Category 3 ops tool. The org's admin settings are a control surface auditors and security architecture care about. This skill makes the audit reproducible: same checks every time, same baseline, structured output.

When to use

• Quarterly admin review (the cadence you'd use for ghas-config-reviewer but at org level).
• Before audit conversations needing documented org posture.
• Troubleshooting an org-wide-policy-induced issue (a team complaining their Action isn't allowed, a user complaining they can't push to a public fork).
• After a major GitHub product change that affects admin settings (model selection, AI policy, residency).

When NOT to use

• Repo-level GHAS configuration → ghas-config-reviewer.
• Workflow security → actions-workflow-hardener.
• Specific user / membership questions → enterprise-audit-log-investigator.
• Copilot-specific policy → copilot-faq-answerer for the question, this skill for the org settings backing it.

Baseline (the org's expected state)

Per category. Severities apply if the current state diverges.

Identity & access

Repository defaults

Security & code scanning

Actions

Copilot

Audit & monitoring

Workflow

1. Authenticate — confirm gh auth status shows access to the target org with admin scope.
2. Inspect via gh api for each baseline category:
   • gh api orgs/{org} — basic settings
   • gh api orgs/{org}/credential-authorizations — SAML enforcement
   • gh api orgs/{org}/scim/v2/Users — SCIM presence
   • gh api orgs/{org}/actions/permissions — Actions allow-list
   • gh api orgs/{org}/actions/runner-groups — runner groups
   • gh api orgs/{org}/copilot/billing — Copilot policy
   • gh api orgs/{org}/security-and-analysis — secret/code scanning defaults
   • gh api enterprises/{enterprise}/audit-log — streaming endpoint
3. Compare each setting against the baseline.
4. Produce findings — structured list per category, presented in chat.
5. If auth fails: emit the manual checklist (below) for a human admin to walk through in the UI.
6. Persist the posture report — after presenting findings (live or manual-fallback), write the full report via vault-writer.write_research to vault/research/github/YYYY-MM-DD-org-audit-{org-slug}.md (frontmatter per research.yml, topic: github) so quarterly reviews can diff against the prior run.

Manual-checklist fallback

When gh api isn't authenticated, output a printable checklist the user can walk through in https://github.com/organizations/{org}/settings. Each item has the UI navigation path + expected state.

Example items:

• [ ] Settings → Authentication security → SAML SSO → enforced for org members
• [ ] Settings → Code security and analysis → Secret scanning → enabled by default for new repos
• [ ] Settings → Actions → General → Actions permissions → "Allow enterprise, and select non-enterprise, actions and reusable workflows"

Output shape

{
  "target_org": "bank-org-name",
  "audit_run_at": "2026-06-20T16:00:00Z",
  "auth_status": "live | manual_fallback",
  "findings": [
    {
      "category": "identity-access",
      "check": "SAML SSO enforced",
      "current_state": "enforced",
      "expected_state": "enforced",
      "verdict": "pass",
      "severity": "info"
    },
    {
      "category": "actions",
      "check": "Allowed Actions list",
      "current_state": "allow-all",
      "expected_state": "restricted allow-list",
      "verdict": "fail",
      "severity": "high",
      "remediation": "Settings → Actions → General → Allow specific actions. Add the org's approved-Actions list.",
      "reference": "vault/research/github/2026-06-20-actions-hardening-post-shai-hulud.md"
    }
  ],
  "summary": {"critical":0,"high":2,"medium":1,"low":0,"info":15,"total":18}
}

Lands at vault/research/github/YYYY-MM-DD-org-audit-{org-slug}.md (workflow step 6).

Composes with

• gh CLI (user-authenticated).
• ghas-config-reviewer — sister skill at the repo level.
• vault-querier — facts for Copilot-specific baseline items.
• vault-writer.write_research — persists the posture report (workflow step 6).

Acceptance test (for step 17 done-criteria)

Produce one audit run output (live or manual fallback). Confirm:

• All 6 baseline categories present in findings.
• Each finding has current_state, expected_state, verdict, severity.
• Summary block has the right counts.
• Failed items include a remediation + reference (vault note or GitHub Docs URL).