CI Forensics
Fetching logs — the correct sequence
1. mcp__github__actions_list method: "list_workflow_runs", branch: <head ref>
2. mcp__github__get_job_logs run_id: <run>, failed_only: true
3. mcp__github__get_job_logs job_id: <job>, return_content: true, tail_lines: 200
Known pitfalls in these tools:
actions_list and actions_get are multiplexed behind a method
discriminator. Calling either without method errors.actions_get also needs resource_id (not run_id) and returns metadata,
not logs. It is the wrong tool for log triage.get_check_run fetches one check run by checkRunId; it has no list mode.
For "is CI green on this sha", use actions_list → list_workflow_runs and
filter by head_sha.list_workflow_runs can exceed the token limit and be spilled to a file.
Parse that file with node -e, do not read it inline.
Failure taxonomy
Classify before changing anything. Fixing an infra flake by editing source code
produces a red PR with extra churn.
| Class |
Signals |
Action |
| Real regression |
Traces to a file/line in the diff; reproduces locally; the test is new or newly touched |
Fix + regression test |
| Flaky |
Same commit sha, different result; timing/ordering/port/network in the trace |
Confirm across ≥3 runs, then quarantine |
| Infra |
Runner OOM, disk full, image pull failure, registry 5xx, action download timeout |
Re-run. Escalate after 2. Never edit code |
| Base broken |
Same job red on the base at a commit predating the branch point |
Report once, wait for recovery |
| Config drift |
Failure in workflow YAML, action inputs, or a missing secret |
Fix the workflow |
| Dependency |
Appeared with no source change; a transitive package published |
Pin or roll back |
Establishing "base broken"
Do not infer it from error text. Check the base branch's own runs for the same
job at a commit predating this branch's point of divergence. Red there too →
base breakage. Otherwise it is yours.
Flaky vs real
| Evidence |
Strength |
| Same commit sha, different results |
Conclusive |
| Passed on a neighbouring run, no relevant change |
Strong |
| Fails only under parallel execution |
Strong — ordering or shared state |
| Fails only on one OS or runtime version |
Medium — environment dependence |
| "It passed when I re-ran it" |
Weak — never sufficient alone |
A test that has never passed is not flaky. It is broken.
Failure rate hints at mechanism: ~50% is usually an async race; ~2% is usually
resource pressure or a real-time dependency.
Sources of nondeterminism
Real time · unseeded randomness · test ordering and shared module state · fixed
ports · shared database rows · reused temp paths · missing awaits · real network
calls · resource pressure.
Never
- Never disable, skip, or
.only a test to make CI pass. .only silently
disables the rest of the file.
- Never re-run a job to make a failure disappear without recording why it failed.
- Never claim CI is fixed without a green run to point at.
See also
actions-authoring — fixing the workflow itself../commands/ci.md — the drive-to-green loop
1---
2name: ci-forensics
3description: This skill should be used when reading CI logs and diagnosing failures — fetching Actions logs through the GitHub MCP, classifying failures by cause, and distinguishing flaky tests from real regressions.
4version: 1.0.0
5trigger_phrases: [CI failing, read CI logs, why is the build red, flaky test, classify failure, drive to green]
6categories: [ci, debugging, github-actions, testing]
7author: github-orchestrator
8created: 2026-08-11
9updated: 2026-08-11
10---
11
12# CI Forensics
13
14## Fetching logs — the correct sequence
15
16```
171. mcp__github__actions_list method: "list_workflow_runs", branch: <head ref>
182. mcp__github__get_job_logs run_id: <run>, failed_only: true
193. mcp__github__get_job_logs job_id: <job>, return_content: true, tail_lines: 200
20```
21
22Known pitfalls in these tools:
23
24- `actions_list` and `actions_get` are multiplexed behind a `method`
25 discriminator. Calling either without `method` errors.
26- `actions_get` also needs `resource_id` (not `run_id`) and returns metadata,
27 not logs. It is the wrong tool for log triage.
28- `get_check_run` fetches **one** check run by `checkRunId`; it has no list mode.
29 For "is CI green on this sha", use `actions_list` → `list_workflow_runs` and
30 filter by `head_sha`.
31- `list_workflow_runs` can exceed the token limit and be spilled to a file.
32 Parse that file with `node -e`, do not read it inline.
33
34## Failure taxonomy
35
36Classify before changing anything. Fixing an infra flake by editing source code
37produces a red PR with extra churn.
38
39| Class | Signals | Action |
40| --- | --- | --- |
41| **Real regression** | Traces to a file/line in the diff; reproduces locally; the test is new or newly touched | Fix + regression test |
42| **Flaky** | Same commit sha, different result; timing/ordering/port/network in the trace | Confirm across ≥3 runs, then quarantine |
43| **Infra** | Runner OOM, disk full, image pull failure, registry 5xx, action download timeout | Re-run. Escalate after 2. Never edit code |
44| **Base broken** | Same job red on the base at a commit predating the branch point | Report once, wait for recovery |
45| **Config drift** | Failure in workflow YAML, action inputs, or a missing secret | Fix the workflow |
46| **Dependency** | Appeared with no source change; a transitive package published | Pin or roll back |
47
48## Establishing "base broken"
49
50Do not infer it from error text. Check the base branch's own runs for the same
51job at a commit predating this branch's point of divergence. Red there too →
52base breakage. Otherwise it is yours.
53
54## Flaky vs real
55
56| Evidence | Strength |
57| --- | --- |
58| Same commit sha, different results | Conclusive |
59| Passed on a neighbouring run, no relevant change | Strong |
60| Fails only under parallel execution | Strong — ordering or shared state |
61| Fails only on one OS or runtime version | Medium — environment dependence |
62| "It passed when I re-ran it" | Weak — never sufficient alone |
63
64A test that has **never** passed is not flaky. It is broken.
65
66Failure rate hints at mechanism: ~50% is usually an async race; ~2% is usually
67resource pressure or a real-time dependency.
68
69## Sources of nondeterminism
70
71Real time · unseeded randomness · test ordering and shared module state · fixed
72ports · shared database rows · reused temp paths · missing awaits · real network
73calls · resource pressure.
74
75## Never
76
77- Never disable, skip, or `.only` a test to make CI pass. `.only` silently
78 disables the rest of the file.
79- Never re-run a job to make a failure disappear without recording why it failed.
80- Never claim CI is fixed without a green run to point at.
81
82## See also
83
84- `actions-authoring` — fixing the workflow itself
85- `../commands/ci.md` — the drive-to-green loop
thelobbi