CI Failure Triage

CI Failure Triage

Remote CI failure diagnosis. Single-snapshot pull of the failing run, commit correlation, ranked hypotheses, fix proposal for user review.

The value of this skill is that it separates diagnosis from fixing. It produces a root-cause hypothesis grounded in evidence (log excerpts + commit SHAs) so the user — or a follow-up debugger dispatch — can act with context instead of guessing.

Language/ecosystem agnostic. The correlation between log evidence and commit history is purely textual (file paths, symbol names, error strings). This skill does NOT interpret language-specific build output — it surfaces the error verbatim and ranks which recent commits most plausibly caused it. Hypothesis patterns (dependency resolution failure, missing env var, test assertion diff, etc.) are inferred from the error text itself, not from a preconfigured language list. Works identically for any project that runs on GitHub Actions regardless of the code's language or framework.

Non-goals

• Does NOT apply fixes. Runtime failures go to the debugger agent, which reproduces, verifies, and fixes.
• Does NOT restart, rerun, or cancel the run.
• Does NOT tail logs interactively. Single snapshot only.
• Does NOT touch workflow YAML. For workflow-file security audits use gha-security-review.

Workflow

IDENTIFY   → Find the failing run
FETCH      → Pull failing job + step logs
CORRELATE  → Match log evidence to recent commits
HYPOTHESIS → Rank top-3 root causes
LOCATE     → Find the source file for the top cause
PROPOSE    → Show a minimal-fix diff (do NOT apply)

Phase 1: IDENTIFY

Get the failing run. Priority order:

1. User named a run ID or URL → use it directly. Extract the numeric ID from URLs of shape https://github.com/<owner>/<repo>/actions/runs/<id>.
2. Otherwise list recent failures on the current branch:

   gh run list --branch "$(git branch --show-current)" --status failure --limit 5
   

   Present the 5 rows. If exactly one row exists, proceed. If multiple, ask the user to pick one before continuing.
3. If the current branch has no failures, try the active PR:

   gh pr checks
   

   Pick the first failing check. If gh pr checks reports no PR, STOP and ask the user for a run ID.

Do NOT invent a run ID. If step 1 and step 2 both yield nothing, STOP and ask.

Phase 2: FETCH

Pull the failing step's output:

gh run view <id> --log-failed

--log-failed captures only failing steps — smaller payload, less noise than --log.

If --log-failed returns empty (some runs fail before any step runs, e.g. YAML parse errors), fall back to:

gh run view <id> --log | tail -200

From the log, extract and record:

• The failing job name.
• The failing step name.
• The final ≤40 lines of log content containing the actual error (error message, stack frames, non-zero exit). Truncate earlier lines. Quote verbatim — do NOT paraphrase.

If the log has no obvious error marker (no Error, FAIL, exit 1, panic, exception, ##[error]), record the last 40 lines as-is and note "no explicit error marker found."

Phase 3: CORRELATE

Tie the log evidence to recent commits.

1. Determine the commit range:
   • If the run is on a PR branch, base = the PR's base branch (find with gh pr view --json baseRefName -q .baseRefName).
   • Otherwise base = origin/main (or origin/master if main does not exist).
2. List commits in range:

   git log --oneline <base>..HEAD
   

3. For each commit (cap at 20 — if more, keep the most recent 20), check if any file it touched appears in the failing log's text (stack frames, paths, module names):

   git show --name-only --pretty=format: <sha>
   

4. Rank commits by correlation strength:
   • HIGH — a file from the commit appears verbatim in the error line or adjacent lines.
   • MEDIUM — a file from the commit appears elsewhere in the captured log.
   • LOW — no overlap; commit is temporally recent only.

Record the top 3 correlated commits with SHA (short), subject, and correlation level.

Phase 4: HYPOTHESIS

Produce a ranked top-3 cause list. Cap at 3. Do NOT speculate beyond the evidence in Phases 2-3.

Format each exactly as:

C1 [confidence HIGH/MED/LOW]: <one-sentence hypothesis>
    evidence: <≤3-line log excerpt>
    commit: <short-sha> <subject>

Confidence rubric:

• HIGH — log names a specific file or symbol that a correlated commit touched.
• MED — error matches a recognised pattern (e.g. dependency resolution failure, missing env var, test assertion diff) and at least one correlated commit is plausibly implicated.
• LOW — error is generic (flaky test, timeout, infra blip) or no correlated commit is plausible.

If ALL three hypotheses would be LOW, report exactly one LOW hypothesis and state: "Evidence is thin — consider rerunning the job to check for flakiness, or paste more context."

Phase 5: LOCATE

For C1 (the most likely cause), find the source file.

Follow the project CLAUDE.md <tool_priority>:

1. If the file is already open (visible in the current editor state), use LSP goToDefinition on the failing symbol.
2. Otherwise, for the failing symbol name, try LSP workspace symbols first.
3. For unfamiliar files, preview structure with gabb_structure before reading — supports Rust, Go, Python, TS/JS, Kotlin, C++, C#, Ruby.
4. Grep as last resort, scoped to the file path from the log:

   Grep pattern="<symbol>" path="<file_from_log>"
   

Read the relevant function body (scope: ±10 lines around the line the log points at). Do NOT read the whole file unless the stack trace demands cross-function context.

Phase 6: PROPOSE

Show a candidate minimal-fix diff — 5 to 20 lines — with reasoning.

Proposed fix for C1:

  <file:line>
  <diff snippet, 5-20 lines>

Reasoning: <1-3 sentences tying the diff to the log evidence>

Do NOT apply the diff. Return for user review.

Offer one explicit follow-up option:

To apply this fix, dispatch the debugger agent with this diff as input — it will reproduce the failure locally, verify the fix flips red to green, and produce a minimal patch. Or ask for a code-reviewer pass first.

Output length cap

Final report ≤ 500 words. Log excerpts ≤ 40 lines each, maximum 2 excerpts total. Diff ≤ 20 lines.

If the report would exceed 500 words, drop Phase 5's surrounding-code context first, then drop C3, then drop C2. Never drop C1.

Report template

## CI Failure Triage — run <id>

**Failing job / step:** <job> / <step>

### Error excerpt (≤40 lines)

Recent commits in scope (..HEAD)

• — correlation: HIGH/MED/LOW

Hypotheses

C1 [HIGH]: ... C2 [MED]: ... C3 [LOW]: ...

Candidate fix for C1

file:line Reasoning: ...

Next step

Apply via debugger, or request a code-reviewer pass first.

---

## Red flags

| Thought | Reality |
|---------|---------|
| "I'll rerun the job to see if it's flaky" | Out of scope. Let the user decide. |
| "I'll just apply the obvious fix" | No. This skill stops at proposal. |
| "The log is long — I'll summarise it" | Quote verbatim. Paraphrasing hides the real error. |
| "All three commits look plausible" | Rank them with evidence. If none correlates, say so. |