CI Integration
Value: Feedback -- CI pipelines produce signals. Deterministic interaction
patterns ensure those signals are received, classified, and acted on correctly.
Undisciplined CI interaction (pushing over failing runs, ignoring flaky tests)
degrades the signal until the pipeline is noise.
Purpose
Teaches disciplined CI/CD interaction: one pending run at a time, structured
failure triage, automated self-healing for mechanical failures, and structured
evidence output for pipeline consumption. Prevents the most common CI failure
mode: pushing again before understanding why the last run failed.
Practices
Push-and-Wait Discipline
One pending CI run at a time. Never push while a run is in progress.
- Push the commit
- Poll or wait for the CI run to complete
- Read the full result before taking any action
- Only after the run completes (pass or fail) may you push again
Do:
- Wait for CI completion before starting new work that would require a push
- Read the complete CI output, not just the status badge
Do not:
- Push a "fix" while the previous run is still pending
- Assume a run will pass and push follow-up commits
- Cancel a run to push a new one (unless the run is clearly stale)
Failure Triage
Classify every CI failure before attempting a fix. The classification
determines the fix strategy.
| Classification |
Signal |
Fix Strategy |
test-failure |
Test assertions fail |
Route to debugging-protocol |
lint-failure |
Linter or formatter errors |
Auto-fix: run formatter, commit, re-push |
build-failure |
Compilation or dependency errors |
Dependency analysis, check lockfiles |
flaky-test |
Test fails then passes on retry without code changes |
Flag and track (see below) |
infra-failure |
Network, runner, or service errors |
Retry (max 2), then escalate |
Read the full CI log to classify. Do not guess from the status alone.
Self-Healing
Mechanical failures that have deterministic fixes should be resolved
automatically:
- Lint/format failures: Run the project's formatter or linter with
auto-fix, commit the result, re-push. This is a single retry -- if the
formatter does not resolve the issue, classify as
build-failure.
- Infra failures: Retry the CI run (max 2 retries). If all retries fail,
escalate to the user with the infra error details.
- Test failures: Never auto-retry. Route to
debugging-protocol for
investigation. A test failure is a signal, not noise.
- Build failures: Check dependency lockfiles, build configuration, and
environment differences. Do not blindly retry.
Flaky Test Detection
A test is flaky if it passes on retry without any code changes.
When detected:
- Flag the test with its name, file, and the failure output from the flaky run
- Record in project memory (
.factory/audit-trail/flaky-tests.json if
factory mode is active, or project notes otherwise)
- Report to the user -- flaky tests erode CI trust and must be addressed
- Do not treat a flaky pass as a real pass for quality gate purposes until
the flakiness is resolved
Structured Output
Every CI interaction produces a CI_RESULT evidence packet:
{
"run_id": "string (CI run identifier)",
"status": "string (passed | failed | cancelled)",
"duration": "number (seconds)",
"failure_type": "string (test-failure | lint-failure | build-failure | flaky-test | infra-failure) -- omit if passed",
"failure_details": "string (summary of failure) -- omit if passed",
"fixes_applied": ["string (description of each auto-fix applied)"],
"retry_count": "number (0 if no retries)"
}
This packet is consumed by pipeline orchestrators and audit trails. Always
produce it, even for passing runs.
Enforcement Note
- Pipeline mode: Gating. CI pass/fail mechanically gates merge.
- Standalone mode: Advisory. The agent self-enforces push-and-wait
discipline.
Hard constraints:
- Push-and-wait:
[H] in pipeline mode, [RP] in standalone
- Flaky pass not counted as real pass:
[H]
Constraints
- Push-and-wait: One pending run means one. Not "one per branch" or
"one per type of change." If a CI run is in progress, do not push
regardless of how unrelated the change seems. The discipline exists
because CI resource contention and merge ordering matter.
- Failure classification: Classify BEFORE attempting any fix. Not "I
tried a fix and it worked, so it must have been a lint failure." The
classification determines the fix strategy, not the other way around.
If you can't confidently classify, investigate further -- don't guess
and fix.
- Flaky test: A flaky test is one that produces different results on
the same code. "Passes on retry without code changes" is the definition.
If you changed the environment, the retry is not evidence of flakiness --
it's evidence of an environment-dependent test. Those are different
problems with different fixes.
Verification
After each CI interaction, verify:
If any criterion is not met, revisit the relevant practice before proceeding.
Dependencies
This skill works standalone for any project with a CI/CD pipeline. It
integrates with:
- debugging-protocol: Test failures route to the debugging protocol for
systematic investigation rather than blind fix attempts
- tdd: CI failures during TDD cycles feed back into the RED-GREEN loop;
a CI test failure means the cycle is not complete
- pipeline: When used within factory mode, the pipeline orchestrator
consumes CI_RESULT packets to evaluate quality gates
Missing a dependency? Install with:
npx skills add jwilger/agent-skills --skill debugging-protocol
1---2name: ci-integration3description: >-4 Deterministic CI/CD interaction patterns with structured evidence output.5 Push-and-wait discipline (one pending run at a time), five-category failure6 triage (test-failure, lint-failure, build-failure, flaky-test, infra-failure)7 with classification-specific fix strategies, self-healing for lint/format/infra8 failures, flaky test detection, and CI_RESULT JSON evidence packets for every9 run. Use when pushing code to CI, triaging CI failures, waiting for pipeline10 results, classifying build errors, dealing with flaky tests, or producing CI11 evidence for quality gates. Triggers on: "CI failed", "push and wait for CI",12 "triage build failure", "flaky test", "CI pipeline", "build broke", "CI13 evidence". NOT for: individual test debugging (use debugging-protocol), code14 review (use code-review).15license: CC0-1.016metadata:17 source: https://skillmd.com/skills/jwilger/ci-integration18 author: jwilger19 version: "1.2.1"20 requires: [debugging-protocol]21 context: [ci-results, git-history]22 phase: ship23 standalone: true24effort: low25---2627# CI Integration2829**Value:** Feedback -- CI pipelines produce signals. Deterministic interaction30patterns ensure those signals are received, classified, and acted on correctly.31Undisciplined CI interaction (pushing over failing runs, ignoring flaky tests)32degrades the signal until the pipeline is noise.3334## Purpose3536Teaches disciplined CI/CD interaction: one pending run at a time, structured37failure triage, automated self-healing for mechanical failures, and structured38evidence output for pipeline consumption. Prevents the most common CI failure39mode: pushing again before understanding why the last run failed.4041## Practices4243### Push-and-Wait Discipline4445One pending CI run at a time. Never push while a run is in progress.46471. Push the commit482. Poll or wait for the CI run to complete493. Read the full result before taking any action504. Only after the run completes (pass or fail) may you push again5152**Do:**53- Wait for CI completion before starting new work that would require a push54- Read the complete CI output, not just the status badge5556**Do not:**57- Push a "fix" while the previous run is still pending58- Assume a run will pass and push follow-up commits59- Cancel a run to push a new one (unless the run is clearly stale)6061### Failure Triage6263Classify every CI failure before attempting a fix. The classification64determines the fix strategy.6566| Classification | Signal | Fix Strategy |67|----------------|--------|--------------|68| `test-failure` | Test assertions fail | Route to `debugging-protocol` |69| `lint-failure` | Linter or formatter errors | Auto-fix: run formatter, commit, re-push |70| `build-failure` | Compilation or dependency errors | Dependency analysis, check lockfiles |71| `flaky-test` | Test fails then passes on retry without code changes | Flag and track (see below) |72| `infra-failure` | Network, runner, or service errors | Retry (max 2), then escalate |7374Read the full CI log to classify. Do not guess from the status alone.7576### Self-Healing7778Mechanical failures that have deterministic fixes should be resolved79automatically:80811. **Lint/format failures:** Run the project's formatter or linter with82 auto-fix, commit the result, re-push. This is a single retry -- if the83 formatter does not resolve the issue, classify as `build-failure`.842. **Infra failures:** Retry the CI run (max 2 retries). If all retries fail,85 escalate to the user with the infra error details.863. **Test failures:** Never auto-retry. Route to `debugging-protocol` for87 investigation. A test failure is a signal, not noise.884. **Build failures:** Check dependency lockfiles, build configuration, and89 environment differences. Do not blindly retry.9091### Flaky Test Detection9293A test is flaky if it passes on retry without any code changes.9495When detected:961. Flag the test with its name, file, and the failure output from the flaky run972. Record in project memory (`.factory/audit-trail/flaky-tests.json` if98 factory mode is active, or project notes otherwise)993. Report to the user -- flaky tests erode CI trust and must be addressed1004. Do not treat a flaky pass as a real pass for quality gate purposes until101 the flakiness is resolved102103### Structured Output104105Every CI interaction produces a `CI_RESULT` evidence packet:106107```json108{109 "run_id": "string (CI run identifier)",110 "status": "string (passed | failed | cancelled)",111 "duration": "number (seconds)",112 "failure_type": "string (test-failure | lint-failure | build-failure | flaky-test | infra-failure) -- omit if passed",113 "failure_details": "string (summary of failure) -- omit if passed",114 "fixes_applied": ["string (description of each auto-fix applied)"],115 "retry_count": "number (0 if no retries)"116}117```118119This packet is consumed by pipeline orchestrators and audit trails. Always120produce it, even for passing runs.121122## Enforcement Note123124- **Pipeline mode**: Gating. CI pass/fail mechanically gates merge.125- **Standalone mode**: Advisory. The agent self-enforces push-and-wait126 discipline.127128**Hard constraints:**129- Push-and-wait: `[H]` in pipeline mode, `[RP]` in standalone130- Flaky pass not counted as real pass: `[H]`131132## Constraints133134- **Push-and-wait**: One pending run means one. Not "one per branch" or135 "one per type of change." If a CI run is in progress, do not push136 regardless of how unrelated the change seems. The discipline exists137 because CI resource contention and merge ordering matter.138- **Failure classification**: Classify BEFORE attempting any fix. Not "I139 tried a fix and it worked, so it must have been a lint failure." The140 classification determines the fix strategy, not the other way around.141 If you can't confidently classify, investigate further -- don't guess142 and fix.143- **Flaky test**: A flaky test is one that produces different results on144 the same code. "Passes on retry without code changes" is the definition.145 If you changed the environment, the retry is not evidence of flakiness --146 it's evidence of an environment-dependent test. Those are different147 problems with different fixes.148149## Verification150151After each CI interaction, verify:152153- [ ] Waited for the previous CI run to complete before pushing154- [ ] Read the complete CI output (not just status)155- [ ] Classified the failure type before attempting a fix156- [ ] Applied the correct fix strategy for the classification157- [ ] Did not retry a test failure (routed to debugging-protocol instead)158- [ ] Lint/format auto-fixes were limited to one attempt159- [ ] Infra retries did not exceed 2160- [ ] Flaky tests were flagged and recorded161- [ ] Produced a CI_RESULT evidence packet162163If any criterion is not met, revisit the relevant practice before proceeding.164165## Dependencies166167This skill works standalone for any project with a CI/CD pipeline. It168integrates with:169170- **debugging-protocol:** Test failures route to the debugging protocol for171 systematic investigation rather than blind fix attempts172- **tdd:** CI failures during TDD cycles feed back into the RED-GREEN loop;173 a CI test failure means the cycle is not complete174- **pipeline:** When used within factory mode, the pipeline orchestrator175 consumes CI_RESULT packets to evaluate quality gates176177Missing a dependency? Install with:178```179npx skills add jwilger/agent-skills --skill debugging-protocol180```