CI CD Patterns

CI/CD Patterns

GitHub Actions workflow optimisation, caching strategies, and pipeline architecture for NodeJS-Starter-V1.

Metadata

Description

Codifies CI/CD patterns for NodeJS-Starter-V1: GitHub Actions workflow architecture with job dependency graphs, multi-layer caching (pnpm, uv, Turborepo, Docker), service containers for integration tests, security scanning pipelines, artifact management, monorepo path filtering, and optional-secret design that works without any external tokens.

When to Apply

Positive Triggers

• Creating or modifying GitHub Actions workflow files
• Adding new CI jobs (test, lint, build, deploy, scan)
• Optimising workflow run times or reducing redundant steps
• Configuring caching for package managers or build outputs
• Setting up deployment pipelines for backend or frontend
• Adding security scanning or dependency review steps

Negative Triggers

• Docker image building patterns (use docker-patterns instead)
• Application-level health endpoints (use health-check instead)
• Local development scripts that do not run in CI
• Infrastructure provisioning (use infrastructure-as-code when available)

Core Principles

The Three Laws of CI/CD

1. Fast Feedback First: Cheap checks (lint, type-check) run before expensive ones (E2E, deploy). Fail fast to save developer time.
2. Cache Everything, Trust Nothing: Cache dependency installs aggressively but always use --frozen-lockfile to ensure reproducibility.
3. Zero Secrets Required: The pipeline must pass with zero configured secrets. Optional features (Snyk, deployment) activate only when their secrets are present.

Pattern 1: Workflow Architecture

Job Dependency Graph

The project's CI pipeline follows a diamond dependency pattern:

dependency-verification
        │
   ┌────┴────┐
   ▼         ▼
backend   frontend     (parallel)
 tests     tests
   │         │
   └────┬────┘
        │
   ┌────┴────┐
   ▼         ▼
 build    e2e-tests    (parallel)
             │
             ▼
      accessibility

Project Reference: ci.yml:14-297 — 6 jobs with needs dependencies.

Job Dependency Rules

Workflow File Organisation

.github/workflows/
├── ci.yml                           # Core CI (tests, lint, build, E2E)
├── security.yml                     # Security scanning (Snyk, Trivy, npm audit)
├── agent-pr-checks.yml              # Agent-generated PR validation
└── examples/
    ├── deploy-backend.yml.example   # DigitalOcean deployment template
    └── deploy-frontend.yml.example  # Vercel deployment template

Rule: Separate concerns into distinct workflow files. CI, security, and deployment should not be in the same file. Use .example suffix for optional deployment workflows that require secrets.

Pattern 2: Caching Strategies

pnpm Cache (Frontend)

- name: Setup pnpm
  uses: pnpm/action-setup@v4
  with:
    version: ${{ env.PNPM_VERSION }}

- name: Setup Node.js
  uses: actions/setup-node@v4
  with:
    node-version: ${{ env.NODE_VERSION }}
    cache: pnpm                      # Built-in pnpm cache support

Project Reference: ci.yml:23-31 — setup-node with cache: pnpm automatically caches the pnpm store.

uv Cache (Backend)

- name: Cache uv dependencies
  uses: actions/cache@v4
  with:
    path: |
      ~/.cache/uv
      apps/backend/.venv
    key: ${{ runner.os }}-uv-${{ hashFiles('apps/backend/pyproject.toml') }}
    restore-keys: |
      ${{ runner.os }}-uv-

Project Reference: ci.yml:87-95 — caches both the uv download cache and the virtualenv.

Turborepo Cache

Turborepo caches task outputs locally in .turbo/. For CI, enable remote caching:

- name: Build all packages
  run: pnpm turbo run build
  env:
    TURBO_TOKEN: ${{ secrets.TURBO_TOKEN }}
    TURBO_TEAM: ${{ secrets.TURBO_TEAM }}

Project Reference: turbo.json:5-8 — build outputs include .next/** (excluding cache) and dist/**.

Cache Key Design

Rule: Always use hashFiles() of the lockfile or dependency manifest as the cache key suffix. Use restore-keys with progressively shorter prefixes for partial cache hits.

Pattern 3: Service Containers

PostgreSQL for Integration Tests

services:
  postgres:
    image: postgres:15-alpine
    env:
      POSTGRES_DB: starter_db
      POSTGRES_USER: starter_user
      POSTGRES_PASSWORD: local_dev_password
    ports:
      - 5432:5432
    options: >-
      --health-cmd pg_isready
      --health-interval 10s
      --health-timeout 5s
      --health-retries 5

Project Reference: ci.yml:60-73 — PostgreSQL service container in backend-tests job.

Service Container Rules

Note: Service containers use localhost:PORT (not Docker network names) because GitHub Actions maps container ports directly to the runner.

Pattern 4: Security Scanning Pipeline

Multi-Layer Security Architecture

┌─────────────────────────────────────────────┐
│ Layer 1: Dependency Review (PR only)        │
│   actions/dependency-review-action@v4       │
│   fail-on-severity: moderate                │
│   deny-licenses: GPL-3.0, LGPL-3.0         │
├─────────────────────────────────────────────┤
│ Layer 2: NPM Audit (no token required)      │
│   pnpm audit --audit-level=high             │
├─────────────────────────────────────────────┤
│ Layer 3: Trivy FS Scan (no token required)  │
│   aquasecurity/trivy-action — severity HIGH │
│   Uploads SARIF to GitHub Security tab      │
├─────────────────────────────────────────────┤
│ Layer 4: Snyk (optional, needs SNYK_TOKEN)  │
│   Frontend + Backend scans                  │
│   if: secrets.SNYK_TOKEN != ''              │
└─────────────────────────────────────────────┘

Project Reference: security.yml:1-200 — all four layers implemented.

Optional Secret Pattern

snyk-frontend:
  name: Snyk Frontend Security Scan (Optional)
  runs-on: ubuntu-latest
  if: ${{ secrets.SNYK_TOKEN != '' }}
  steps:
    # ...

Rule: Use if: ${{ secrets.SECRET_NAME != '' }} to conditionally run jobs that need tokens. This ensures the workflow passes when no secrets are configured (zero-barrier principle).

Pattern 5: Artifact Management

Upload Pattern

- name: Upload coverage report
  if: always()
  uses: actions/upload-artifact@v4
  with:
    name: backend-coverage
    path: |
      apps/backend/coverage.xml
      apps/backend/htmlcov/
    retention-days: 30

Project Reference: ci.yml:115-123 — coverage uploads in backend-tests job.

Artifact Naming Conventions

Rule: Always use if: always() on artifact upload steps. Test results and coverage reports are most valuable when tests fail, not when they pass.

Pattern 6: Monorepo Path Filtering

Deploy Only Changed Services

on:
  push:
    branches: [main]
    paths:
      - "apps/backend/**"

Project Reference: deploy-backend.yml.example:5-7 — only triggers on backend changes.

Path Filter Patterns

Rule: Deployment workflows should use paths filters to avoid unnecessary deployments. CI and security workflows should run on all changes.

Pattern 7: Agent PR Validation

Agent-Specific Quality Gates

The project has a dedicated workflow for AI-generated PRs:

on:
  pull_request:
    branches: [main, develop]

jobs:
  detect-agent-pr:
    steps:
      - name: Check if agent PR
        run: |
          if [[ "${{ github.head_ref }}" == feature/agent-* ]]; then
            echo "is_agent=true" >> $GITHUB_OUTPUT
          fi

Project Reference: agent-pr-checks.yml:1-174 — validates agent metadata, runs quality checks, scans for secrets/debug code, posts PR status comment.

Agent PR Checks

Pattern 8: Environment Variable Management

Centralised Version Pinning

env:
  NODE_VERSION: '20'
  PNPM_VERSION: 9
  PYTHON_VERSION: '3.12'

Project Reference: ci.yml:9-13 — version matrix defined at workflow level.

Rule: Define tool versions once at the workflow env level, then reference with ${{ env.TOOL_VERSION }} in all jobs. This ensures consistency and makes upgrades a single-line change.

Secret Tiers

Project Reference: .github/SECRETS.md:1-60 — full secrets documentation with setup instructions.

Pattern 9: Verification Scripts

Reusable Shell Functions

# scripts/dependency-checks.sh — sourced by CI and verify.sh
source scripts/dependency-checks.sh
if ! check_lockfile_integrity; then
  echo "::error::Lockfile integrity check failed"
  exit 1
fi

Project Reference: scripts/dependency-checks.sh:10-50 — lockfile integrity checks. ci.yml:34-39 — used in dependency-verification job.

Rule: Extract verification logic into reusable shell scripts in scripts/. Source them in both CI workflows and local verification commands (pnpm run verify). This ensures local and CI validation are identical.

Anti-Patterns

Checklist

Before merging CI/CD changes:

• Jobs parallelised where independent (tests, scans)
• Expensive jobs gated behind cheap checks (needs)
• --frozen-lockfile used for all package installs
• Package manager caches configured (pnpm, uv)
• Service containers have healthcheck options
• Artefact uploads use if: always() and retention-days
• Optional features use if: secrets.X != '' guard
• Tool versions centralised in workflow env block
• Deployment workflows use paths filter
• Security scans run on schedule (weekly) in addition to push/PR

Response Format

When applying this skill, structure implementation as:

### CI/CD Implementation

**Workflow**: [ci / security / deploy / custom]
**Trigger**: [push / pull_request / schedule / workflow_dispatch]
**Jobs**: [job dependency graph]
**Caching**: [pnpm / uv / Turbo / Docker layers]
**Secrets Required**: [none / optional / required]
**Path Filter**: [all / specific paths]