# Validating Production Readiness

> The lenne.tech fullstack production-readiness checklist. Defines the eight pillars (configuration and secrets, observability, health and lifecycle, security hardening, data durability, resilience under load, deployment hygiene, runbook and rollback) with file/line evidence, severity classification, and a machine-parseable report block. Activates on "production ready", "produktionsreif", "Go-Live", "kann das live", or when a release must be gated before shipping. NOT for OWASP code-level security review (use the security-reviewer agent). NOT for npm dependency audits (use maintaining-npm-packages).

- Skill: `lennetech/validating-production-readiness` (Agent Skill)
- Install (CLI): `npx skillmds@latest add lennetech/validating-production-readiness`
- Raw SKILL.md: https://api.skillmd.com/api/skills/lennetech/validating-production-readiness/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: lenneTech (https://skillmd.com/u/lennetech)
- Updated: 2026-09-10
- Page: https://skillmd.com/skills/lennetech/validating-production-readiness

---


# Validating Production Readiness

This skill is the **single source of truth** for the production-readiness gate in the lt-stack. Every consumer applies the same eight pillars, the same severity classification, and produces the same report block — so multiple runs across time can be compared.

> **Goal:** Decide, with evidence, whether the system is safe to deploy to production. Anything Critical is a hard blocker. Anything Major must be tracked. Minor findings are advisory.

## When to Use This Skill

| Caller | Phase | Trigger |
|--------|-------|---------|
| `/lt-dev:production-ready` | Phase 4 | Hard release gate |
| `lt-dev:production-readiness-orchestrator` | Phase 4 | Owns the audit + remediation loop |
| `lt-dev:devops-reviewer` | (cross-reference) | Cites this checklist when flagging deployment-relevant gaps |
| Manual user invocation | Pre-deploy | Final sign-off before tagging a release |

## Severity Classification

| Severity | Meaning | Effect on the release |
|----------|---------|------------------------|
| **Critical** | Loss of data, security breach, or undetectable downtime is plausible | **BLOCK** the release until fixed |
| **Major** | Operability is degraded; recovery requires manual intervention | Tracked finding; release may proceed only with explicit owner sign-off |
| **Minor** | Cosmetic, missing nice-to-have, or hygienic gap with no operational impact | Advisory only |

A pillar with even a single Critical finding makes the whole release **NOT READY**.

## The Eight Pillars

Each pillar lists the concrete checks to perform, the file paths to inspect, and the typical fix when the check fails.

### Pillar 1 — Configuration & Secrets

| Check | How | Severity if missing |
|-------|-----|----------------------|
| Env validation at boot | `projects/api/src/config.env.ts` (or equivalent) parses with class-validator / Valibot and crashes on missing required keys | Critical |
| `.env*` excluded from VCS | `.gitignore` covers `.env`, `.env.*`, **except** `.env.example` | Critical |
| `.env.example` exists & is current | All keys read by code are present with placeholder values | Major |
| Secrets are runtime-injected (not baked) | Production Dockerfile does NOT `COPY .env`; secrets come from CI vault | Critical |
| `NODE_ENV` taxonomy respected | `e2e` (local tests), `ci` (pipeline tests), `develop` (dev server), `test` (customer staging — NOT a test env), `production` (live) | Major |
| Per-env DB URI / database name | `nsc.mongoose.uri` ends with the env-specific DB (no shared "test" DB across envs) | Critical |
| Default-credentials guard | Production refuses to boot with `password === 'admin'` / `JWT_SECRET === 'change-me'` etc | Critical |

### Pillar 2 — Observability & Logging

| Check | How | Severity if missing |
|-------|-----|----------------------|
| Structured logs (JSON) in production | `pino-http` or NestJS Logger configured with JSON transport when `NODE_ENV=production` | Major |
| No `console.log` in shipped server code | `grep -nR "console\.log" projects/api/src` returns 0 (`console.error` for boot failures is acceptable) | Major |
| Request ID / correlation ID | Middleware emits and propagates `x-request-id` to outgoing calls | Major |
| Error tracking wired up | Sentry (or equivalent) initialised on both API and App when DSN is set | Major |
| Metrics endpoint or APM | `/metrics` (Prometheus) OR APM agent (NewRelic/Datadog) instrumented | Minor |
| Frontend error boundary | Nuxt `error.vue` renders a useful message and reports to Sentry | Major |
| Log levels controlled by env | `LOG_LEVEL` env var read; defaults to `info` in prod, `debug` in dev | Minor |

### Pillar 3 — Health & Lifecycle

| Check | How | Severity if missing |
|-------|-----|----------------------|
| Liveness endpoint | `GET /health` (or `/meta`) returns 200 without DB dependency, suitable for k8s liveness probe | Critical |
| Readiness endpoint | `GET /ready` performs DB ping; returns 503 when DB is unreachable | Major |
| Graceful shutdown | NestJS `app.enableShutdownHooks()` enabled; in-flight requests drain before process exit | Critical |
| Frontend SSR health | `GET /` returns 200 in SSR with mocked-or-real backend; no unhandled rejections in server log | Major |
| Docker `HEALTHCHECK` declared | Production Dockerfile includes `HEALTHCHECK` instruction calling `/health` | Major |
| Process isolation | Container runs as non-root user (`USER node` or similar) | Major |

### Pillar 4 — Security Hardening

| Check | How | Severity if missing |
|-------|-----|----------------------|
| Helmet (or equivalent) on API | `app.use(helmet())` for REST mode; CSP set explicitly | Major |
| CORS allowlist | `cors.origin` is an explicit list, NOT `*` in production | Critical |
| HTTPS enforced | Reverse proxy / Nuxt config redirects http → https in production | Critical |
| Cookies secure | Better-Auth cookies have `secure: true`, `httpOnly: true`, `sameSite: 'lax'` (or stricter) in production | Critical |
| Rate limiting on auth | `@nestjs/throttler` (or equivalent) on `/auth/*`, `/forgot-password`, etc | Major |
| Restricted decorators on every controller | `@Restricted` / `@Roles` / `securityCheck()` present per backend-dev guidelines | Critical |
| `pnpm audit` clean (or audit ladder exhausted) | Per `running-check-script` skill rules | Major |
| No secrets in client bundle | `grep -nR "JWT_SECRET\|MONGO_URI\|API_KEY" projects/app/.output/` returns 0 after build | Critical |

### Pillar 5 — Data Durability

| Check | How | Severity if missing |
|-------|-----|----------------------|
| Indexes for hot queries | All queries seen in load-test profiler (Pillar 6 / k6 phase) hit an index | Major |
| Migrations / seed scripts deterministic | `pnpm run migrate` is idempotent; replaying it on production data does not corrupt | Critical |
| Backup strategy documented | `README.md` or `docs/runbook.md` describes backup cadence + restore drill | Major |
| No destructive migrations on boot | Server does NOT run `dropDatabase` / `dropCollection` on startup in production | Critical |
| Soft-delete or audit trail where data loss matters | Models flagged as critical have `deletedAt` / version trail | Minor |
| Connection retry / backoff | Mongoose connect uses exponential backoff, not infinite tight loop | Major |

### Pillar 6 — Resilience Under Load

The k6 results from the `running-load-tests-with-k6` skill feed directly into this pillar. Don't re-run k6 here — read the report block.

| Check | How | Severity if missing |
|-------|-----|----------------------|
| k6 load scenario PASSED | `tests/load/summary-load.json` thresholds all PASS | Critical |
| k6 baseline regression < 15% | p95 delta vs `tests/load/baselines/` within bounds | Major |
| No memory leak in soak (when run) | `summary-soak.json` shows flat memory at end | Major |
| Connection pool not exhausted | Mongoose log shows < 80% of `maxPoolSize` at peak | Major |
| Timeouts everywhere | All HTTP / DB calls have explicit timeouts (no infinite waits) | Major |
| Idempotency keys on critical writes | Payment / email / external write endpoints have idempotency-key handling | Major |

### Pillar 7 — Deployment Hygiene

| Check | How | Severity if missing |
|-------|-----|----------------------|
| Dockerfile multi-stage | Builder + runtime stages; runtime image contains only built artefacts + prod deps | Major |
| Image base pinned | `FROM node:22.11.0-alpine` (concrete version), not `node:latest` | Major |
| `.dockerignore` complete | `.git`, `node_modules`, `tests/`, `.env*` excluded | Major |
| CI builds from main with tag | Tag-triggered pipeline builds the production image; never `latest`-only | Major |
| Build is reproducible | Lockfiles (`pnpm-lock.yaml`) committed; CI uses `--frozen-lockfile` | Major |
| Image scan in CI | `trivy` / `grype` / GitLab container scan stage exists | Major |
| Image size reasonable | Final image < 500 MB (alpine) / < 1 GB (debian-slim) | Minor |
| Build identity baked (drift detection) | Both images receive `APP_VERSION_COMMIT` (= CI commit SHA) as a build arg; `GET /meta.commit` returns the real SHA (not `unknown`) and the App `/app/admin/system` compares App vs. API to catch a partial / stale rollout | Major |

### Pillar 8 — Runbook & Rollback

| Check | How | Severity if missing |
|-------|-----|----------------------|
| Runbook exists | `docs/runbook.md` (or equivalent) describes deploy, rollback, common incidents | Major |
| Rollback path documented | Tagged previous image is deployable in < 5 min | Critical |
| On-call ownership clear | README or runbook names the responsible team / Slack channel | Minor |
| Migration reversibility | Each migration has a documented rollback (or is annotated as one-way) | Major |
| Feature flags for risky changes | Risky paths gated by env var or runtime flag | Minor |
| Monitoring alerts configured | At least: API down, error rate > 5%, p95 > 1s, DB connection lost | Major |

## Audit Procedure

For each pillar, in order:

1. **Inspect** the listed files / configs / endpoints. Use `Read`, `Grep`, `Glob` only — no destructive Bash.
2. **Classify each finding** as Critical / Major / Minor with file:line evidence.
3. **Record the verdict per pillar** as `PASS` (no findings ≥ Major), `WARN` (Major only), `FAIL` (any Critical).
4. **Roll up the global verdict:** `READY` (all pillars PASS), `READY-WITH-NOTES` (≥1 WARN, 0 FAIL), `NOT-READY` (any FAIL).

## Remediation Loop (when called from production-ready command)

When called from `/lt-dev:production-ready`, the orchestrator MUST attempt remediation for each Critical / Major finding:

| Finding type | Default fix |
|--------------|-------------|
| Missing `.env.example` key | Add the placeholder key with a comment describing the value |
| Missing `enableShutdownHooks` | Add to `main.ts` |
| Missing `helmet` / CORS allowlist | Add to bootstrap module |
| Missing `HEALTHCHECK` in Dockerfile | Add `HEALTHCHECK CMD wget -qO- http://localhost:3000/health || exit 1` |
| Missing rate limiting on `/auth/*` | Add `@Throttle` decorator with sane defaults |
| `console.log` in shipped code | Replace with NestJS `Logger` / `pino` instance |
| Missing index | Add `@Index` decorator + migration to backfill |
| Missing runbook | Scaffold `docs/runbook.md` from the runbook template |

If a finding cannot be auto-fixed (architectural change, missing infra access, business decision), classify it as **needs-human** and surface it in the report rather than guessing.

After every remediation pass, re-run the relevant pillar check.

## Report Block (canonical)

Every consumer ends Phase 4 with this block:

```
### Production Readiness Report

| Pillar | Verdict | Critical | Major | Minor | Auto-fixed |
|--------|---------|----------|-------|-------|-------------|
| 1. Configuration & Secrets | <PASS|WARN|FAIL> | <n> | <n> | <n> | <n> |
| 2. Observability & Logging | …
| 3. Health & Lifecycle      | …
| 4. Security Hardening      | …
| 5. Data Durability         | …
| 6. Resilience Under Load   | …
| 7. Deployment Hygiene      | …
| 8. Runbook & Rollback      | …

Global verdict: <READY|READY-WITH-NOTES|NOT-READY>
Auto-fixed total: <n>
Needs-human findings: <n>
Blocking issues:
- <pillar/severity/file:line> — short description
```

## Cross-Skill References

- **Code-level security:** `general-frontend-security` (frontend XSS/CSRF) and `lt-dev:security-reviewer` agent (OWASP-aligned diff review)
- **DevOps configuration:** `lt-dev:devops-reviewer` agent (Docker / Compose / CI/CD specifics)
- **Load resilience evidence:** `running-load-tests-with-k6` (k6 results feed Pillar 6)
- **Runnability gate:** `running-check-script` (must already be GREEN before this skill runs)
- **Server lifecycle for health checks:** `managing-dev-servers`