CI Smoke Needs Real Deps

CI Smoke Tests Need Real Dependencies

A smoke test that just runs node out/main.mjs && curl localhost:PORT/health is not a smoke test if the binary can't reach a real database. When stacks change (Mongo → Postgres, sqlite → mysql, in-memory → Redis, …), the release pipeline often gets forgotten while ci.yml is updated. Symptom: tag push triggers release workflow, build job fails with connection refused or missing-env throws.

When to use

• Release workflow fails at "smoke test" / "test bundle" / "boot test" step
• Error says Connection refused, MONGO_URL is required, SNOWFLAKE_WORKER_ID is required, cannot connect to ..., or similar after a recent stack migration
• The same project's ci.yml (PR checks) passes but release.yml fails — they've diverged
• Adding a new external dependency that the smoke test needs

When NOT to use

• Smoke test is genuinely failing (real bug in the code) — fix the code, don't paper over it
• The dep would slow CI to the point of being useless — consider whether the smoke is the wrong level of test (use unit tests instead)

Diagnostic checklist

When release.yml smoke fails post-migration, check what ci.yml does for its smoke and align:

1. What services: does ci.yml declare? (postgres? mysql? redis?)
2. What env: does ci.yml inject on the smoke step?
3. Is there a migration runner or MIGRATIONS_DIR env that ci.yml passes but release.yml doesn't?
4. Does the bundled binary auto-run migrations on boot, or does it need an explicit migrate step first?

If (1) or (2) differ from release.yml, that's the bug.

The Pattern

GitHub Actions example (mirror your ci.yml smoke job):

build:
  runs-on: ubuntu-latest
  services:
    postgres:                       # match the stack the binary actually expects
      image: postgres:16-alpine     # pin major; -alpine is fine for tests
      env:
        POSTGRES_USER: app
        POSTGRES_PASSWORD: app
        POSTGRES_DB: app
      ports:
        - 5432:5432
      options: >-
        --health-cmd "pg_isready -U app -d app"
        --health-interval 10s
        --health-timeout 5s
        --health-retries 5
    redis:
      image: redis:7-alpine
      ports:
        - 6379:6379
  steps:
    - uses: actions/checkout@v6
    - uses: ./.github/actions/setup-node
      with: { node-version: '22.x' }
    - run: pnpm bundle
    - name: Smoke test bundle
      env:
        # Every env var the binary requires at boot.
        # Match exactly what ci.yml uses — do not invent new values.
        SNOWFLAKE_WORKER_ID: 1
        PG_HOST: 127.0.0.1
        PG_PORT: 5432
        PG_USER: app
        PG_PASSWORD: app
        PG_DATABASE: app
        REDIS_HOST: 127.0.0.1
        REDIS_PORT: 6379
        JWT_SECRET: smoke-test-only-not-a-real-secret
        # If the binary expects migrations on disk, point at the source tree
        MIGRATIONS_DIR: ${{ github.workspace }}/path/to/migrations
      run: bash scripts/workflow/test-server.sh

Why this matters more than it sounds

A release workflow that builds + ships without a real-deps smoke test will publish a Docker image that crashes on first boot in production. The "successful" tag push gives false confidence. The fix is cheap (one health-checked service container + a handful of env vars). The cost of skipping is shipping a broken image.

Aligning two workflows

Use a quick diff to keep ci.yml and release.yml in sync on the smoke section:

diff <(yq '.jobs.test.services' ci.yml) \
     <(yq '.jobs.build.services' release.yml)
diff <(yq '.jobs.test.steps[] | select(.name == "Test Bundle Server")' ci.yml) \
     <(yq '.jobs.build.steps[] | select(.name == "Test Bundle Server")' release.yml)

When they drift, fix release.yml — ci.yml runs more often and is usually the more current of the two.

Common Mistakes