cicd-pipeline-hardening
A CI/CD pipeline runs your code and has credentials to your infrastructure — it's both a productivity tool and one of the highest-value attack surfaces you own. This skill builds one that is fast (cached, parallel, fails early), reproducible (pinned, hermetic), and hard to abuse (least-privilege, pinned third-party actions, provenance on artifacts).
Use when
- Setting up CI for a new repo.
- Builds are slow, flaky, or wildly non-reproducible.
- A security review flagged the pipeline (over-privileged tokens, unpinned actions, secrets in logs).
Stage ordering — fail cheap, fail first
Order stages by cost so the pipeline fails on the cheapest signal:
- Lint + format check (seconds) — reject style/obvious errors before spending compute.
- Typecheck (fast) — catch contract breaks.
- Unit tests (fast, parallelizable).
- Build — produce the artifact once; later stages reuse it, never rebuild.
- Integration/e2e tests (slow) — only after the fast gates are green.
- Security scans (SAST, dependency audit, secret scan) — can run in parallel with tests.
- Publish/deploy — only on the default branch, only after everything above passes.
Run independent stages in parallel; make expensive stages depend on cheap ones passing. A PR that fails lint should never reach e2e.
Reproducibility
- Pin everything. Lockfiles committed and
--frozen/ci installs (npm ci, pip install --require-hashes, etc.). Pin the runner image and toolchain versions — "latest" is a time bomb.
- Build once, promote the same artifact. Build the image/bundle a single time, then promote that exact artifact through staging → prod. Never rebuild per environment — a rebuild can produce a different artifact than the one you tested.
- Hermetic where you can. Restrict network during build; vendor or cache dependencies so a flaky registry doesn't fail your build and an outage doesn't block a deploy.
Caching (the biggest speed win)
- Cache the dependency store (
~/.npm, ~/.cache/pip, Go/Cargo caches) keyed on the lockfile hash — restores on cache hit, rebuilds only when deps change.
- Cache build outputs where the toolchain supports it (Turborepo/Nx remote cache, Docker layer cache with
cache-from/cache-to).
- Key caches precisely: too-broad keys serve stale caches; too-narrow keys never hit. Include the lockfile hash + OS + tool version in the key.
Supply-chain hardening (this is where pipelines get owned)
| Risk |
Fix |
| Third-party action runs arbitrary code with your token |
Pin actions to a full commit SHA, not a tag — tags are mutable and can be re-pointed at malicious code. |
Over-privileged GITHUB_TOKEN |
Set permissions: to least privilege (default read-all), grant write only to the specific job that needs it. |
| Long-lived cloud credentials in secrets |
Use OIDC federation — short-lived tokens minted per run, no static cloud keys stored in CI. |
| Untrusted PR code reading secrets |
pull_request from forks runs without secrets by default — keep it that way; never use pull_request_target to hand secrets to fork code. |
| Secrets printed to logs |
Never echo a secret; rely on the platform's masking; audit steps that might interpolate a secret into output. |
| Tampered build artifacts |
Generate provenance/SLSA attestation and sign artifacts (cosign); verify signatures at deploy. |
| Malicious dependency |
Run a dependency audit + secret scan in CI; fail on known-critical vulns. |
Deploy gates
- Protect the default branch: required status checks, required review, no direct pushes.
- Environments with approvals for production — a human (or a policy) gates the prod deploy; staging can be automatic.
- Deploys are reversible: every deploy has a rollback path (previous artifact promotable, or automated rollback on health-check failure). Never a one-way door.
- Smoke-test after deploy and auto-rollback on failure, rather than discovering the breakage from users.
- Concurrency control so two deploys don't race the same environment.
Procedure
- Order stages cheap→expensive; parallelize the independent ones; gate expensive on cheap.
- Pin runner, toolchain, actions (by SHA); commit lockfiles; use frozen installs.
- Add dependency caching keyed on lockfile hash; enable layer/remote build cache.
- Set
permissions: least-privilege per job; move cloud creds to OIDC; verify no secrets reach fork PRs or logs.
- Build the artifact once; promote it through environments; sign + attest.
- Add branch protection, prod approval gate, post-deploy smoke test, and a rollback path.
- Verify: a red lint fails in seconds; a clean run is cached and fast;
permissions and pins pass a security scan.
Definition of done
- Fails on the cheapest failing signal; independent stages run in parallel.
- Fully pinned (runner, tools, actions-by-SHA, lockfiles); one artifact promoted, not rebuilt.
- Least-privilege tokens; OIDC for cloud; no secrets exposed to fork PRs or logs.
- Artifacts signed/attested; deploys gated, smoke-tested, and reversible.
1---2name: cicd-pipeline-hardening3description: Design or harden a CI/CD pipeline so it's fast, reproducible, and can't be turned into a supply-chain attack vector. Use when setting up CI for a repo, when builds are slow/flaky, or when a security review flags the pipeline. Covers stage ordering, caching, least-privilege tokens, pinned actions, artifact provenance, and safe deploy gates. GitHub Actions-first, portable.4---56# cicd-pipeline-hardening78A CI/CD pipeline runs your code *and* has credentials to your infrastructure — it's both a productivity tool and one of the highest-value attack surfaces you own. This skill builds one that is fast (cached, parallel, fails early), reproducible (pinned, hermetic), and hard to abuse (least-privilege, pinned third-party actions, provenance on artifacts).910## Use when11- Setting up CI for a new repo.12- Builds are slow, flaky, or wildly non-reproducible.13- A security review flagged the pipeline (over-privileged tokens, unpinned actions, secrets in logs).1415## Stage ordering — fail cheap, fail first16Order stages by cost so the pipeline fails on the cheapest signal:171. **Lint + format check** (seconds) — reject style/obvious errors before spending compute.182. **Typecheck** (fast) — catch contract breaks.193. **Unit tests** (fast, parallelizable).204. **Build** — produce the artifact once; later stages reuse it, never rebuild.215. **Integration/e2e tests** (slow) — only after the fast gates are green.226. **Security scans** (SAST, dependency audit, secret scan) — can run in parallel with tests.237. **Publish/deploy** — only on the default branch, only after everything above passes.2425Run independent stages in parallel; make expensive stages depend on cheap ones passing. A PR that fails lint should never reach e2e.2627## Reproducibility28- **Pin everything.** Lockfiles committed and `--frozen`/`ci` installs (`npm ci`, `pip install --require-hashes`, etc.). Pin the runner image and toolchain versions — "latest" is a time bomb.29- **Build once, promote the same artifact.** Build the image/bundle a single time, then promote *that exact artifact* through staging → prod. Never rebuild per environment — a rebuild can produce a different artifact than the one you tested.30- **Hermetic where you can.** Restrict network during build; vendor or cache dependencies so a flaky registry doesn't fail your build and an outage doesn't block a deploy.3132## Caching (the biggest speed win)33- Cache the dependency store (`~/.npm`, `~/.cache/pip`, Go/Cargo caches) keyed on the lockfile hash — restores on cache hit, rebuilds only when deps change.34- Cache build outputs where the toolchain supports it (Turborepo/Nx remote cache, Docker layer cache with `cache-from`/`cache-to`).35- Key caches precisely: too-broad keys serve stale caches; too-narrow keys never hit. Include the lockfile hash + OS + tool version in the key.3637## Supply-chain hardening (this is where pipelines get owned)38| Risk | Fix |39| --- | --- |40| Third-party action runs arbitrary code with your token | **Pin actions to a full commit SHA**, not a tag — tags are mutable and can be re-pointed at malicious code. |41| Over-privileged `GITHUB_TOKEN` | Set `permissions:` to least privilege (default `read-all`), grant `write` only to the specific job that needs it. |42| Long-lived cloud credentials in secrets | Use **OIDC federation** — short-lived tokens minted per run, no static cloud keys stored in CI. |43| Untrusted PR code reading secrets | `pull_request` from forks runs without secrets by default — keep it that way; never use `pull_request_target` to hand secrets to fork code. |44| Secrets printed to logs | Never `echo` a secret; rely on the platform's masking; audit steps that might interpolate a secret into output. |45| Tampered build artifacts | Generate provenance/SLSA attestation and sign artifacts (cosign); verify signatures at deploy. |46| Malicious dependency | Run a dependency audit + secret scan in CI; fail on known-critical vulns. |4748## Deploy gates49- **Protect the default branch:** required status checks, required review, no direct pushes.50- **Environments with approvals** for production — a human (or a policy) gates the prod deploy; staging can be automatic.51- **Deploys are reversible:** every deploy has a rollback path (previous artifact promotable, or automated rollback on health-check failure). Never a one-way door.52- **Smoke-test after deploy** and auto-rollback on failure, rather than discovering the breakage from users.53- Concurrency control so two deploys don't race the same environment.5455## Procedure561. Order stages cheap→expensive; parallelize the independent ones; gate expensive on cheap.572. Pin runner, toolchain, actions (by SHA); commit lockfiles; use frozen installs.583. Add dependency caching keyed on lockfile hash; enable layer/remote build cache.594. Set `permissions:` least-privilege per job; move cloud creds to OIDC; verify no secrets reach fork PRs or logs.605. Build the artifact once; promote it through environments; sign + attest.616. Add branch protection, prod approval gate, post-deploy smoke test, and a rollback path.627. Verify: a red lint fails in seconds; a clean run is cached and fast; `permissions` and pins pass a security scan.6364## Definition of done65- Fails on the cheapest failing signal; independent stages run in parallel.66- Fully pinned (runner, tools, actions-by-SHA, lockfiles); one artifact promoted, not rebuilt.67- Least-privilege tokens; OIDC for cloud; no secrets exposed to fork PRs or logs.68- Artifacts signed/attested; deploys gated, smoke-tested, and reversible.
omonuj