# Shipping And Release

> Shipping & Release

- Skill: `sanjeevrg89/shipping-and-release` (Agent Skill, multi-file: 5 files)
- Install (CLI): `npx skillmds@latest add sanjeevrg89/shipping-and-release`
- Raw SKILL.md: https://api.skillmd.com/api/skills/sanjeevrg89/shipping-and-release/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- Author: sanjeevrg89 (https://skillmd.com/u/sanjeevrg89)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/sanjeevrg89/shipping-and-release

---


# Shipping & Release

Apply the judgment of an engineer who has shipped to production for years and been paged for the
fallout: who has watched a "tiny" config change take down inference for a region, rolled back a model
that passed offline eval but tanked the online quality metric, and learned the hard way that an
untested rollback is not a rollback. The whole discipline reduces to one rule: **ship small, ship
reversibly, watch it.** Never big-bang an irreversible change into production.

## How to use this skill

1. **Read `shipping-and-release-guide.md`** in this directory — the full process and reference. Apply
   it to the rollout at hand. For a concrete model-rollout plan, a pre-ship readiness checklist, and a
   rollback-runbook template to imitate, read **`examples.md`**.
2. This is the **Ship** stage. Code is already written, reviewed ([[code-review-discipline]]) and
   verified ([[verification-and-debugging]]); deployment-pattern *mechanics* and ML delivery pipelines
   are in [[mlops-lifecycle]]. This skill is the *discipline* of how to release without breaking prod.
3. Match the team's existing release tooling and conventions; apply the safety gates regardless. If the
   environment can't support a gate (no rollback, no monitoring), that is a finding — surface it, don't
   skip it silently.

## Essentials (full detail in `shipping-and-release-guide.md`)

- **Ship small, ship reversibly, watch it.** Three properties of every safe release: the change is
  small (bounded blast radius), it can be undone fast, and you are watching the right signals while it
  ramps. Missing any one → you are gambling, not shipping.
- **Progressive delivery, always.** Roll out to a slice before everyone: canary by % of traffic,
  blue-green for instant switch-back, shadow/dark launch for zero-user-impact validation, feature flags
  to decouple deploy from release. Default to canary for anything user-facing.
- **Model rollouts are riskier than code rollouts** — offline eval ≠ online behavior. Always
  **shadow → small canary % → staged ramp → 100%**, gated on online quality/latency/cost, with
  **champion/challenger** comparison. Never swap a model 0→100%. (Mechanics: [[mlops-lifecycle]],
  [[gke-inference-gateway]].)
- **The pre-ship readiness gate is a hard gate, not a checklist you wave through.** Before ramping:
  rollback plan **tested** (executed in staging, not just written), monitoring/alerts/SLOs **live
  before** the first user sees the change ([[ml-observability-monitoring]]), capacity/quota confirmed
  ([[autoscaling-kubernetes]], [[gke-master]]), runbook written, on-call aware, blast radius bounded,
  data migrations reversible. No-go on any → don't ramp.
- **A rollback that has never been executed is a hope, not a plan.** Test it before you need it.
- **Monitoring goes live before the ramp, not after.** "We'll add monitoring later" means you ramp
  blind and find out from users.
- **The ramp loop:** deploy to a slice → watch errors, latency/TTFT, eval/quality, cost against
  explicit abort criteria → proceed to the next step or **roll back**. Bake at each step long enough to
  see real traffic. Don't skip steps because the last one looked fine.
- **When it goes wrong: roll back first, debug later.** Mitigation precedes diagnosis. Restore the
  good state, *then* investigate the bad one ([[verification-and-debugging]]).
- **Reversible migrations.** Expand/contract (additive first, destructive later, after the new code is
  proven). Never deploy code and an irreversible schema change in the same irreversible step.
- **Change management for infra:** GitOps (Git as source of truth, declarative, auditable, revert =
  rollback). Do risky cluster changes in a maintenance window, off-hours, with on-call online.
- **Blameless incident review** after anything that went sideways. Fix the system and the process, not
  the person. Feed the learning back into the readiness gate.
- **Don't ship into a hole.** No Friday-evening ship of a risky change with no on-call. No ship while
  you're the only one who understands it and you're about to be unreachable.

## Related skills

- `[[engineering-lifecycle]]` — the umbrella; Ship is one stage. Start there for the whole arc.
- `[[code-review-discipline]]` — the gate *before* ship; nothing ramps that wasn't reviewed.
- `[[verification-and-debugging]]` — verify before ship; debug *after* you've rolled back.
- `[[mlops-lifecycle]]` — deployment-pattern mechanics and ML delivery/CD pipelines (the *how*).
- `[[ml-observability-monitoring]]` — the signals you watch during the ramp; must be live first.
- `[[gke-inference-gateway]]` — traffic splitting / model routing for canary and shadow on GKE.
- `[[autoscaling-kubernetes]]` — capacity/quota headroom so the ramp doesn't starve.
- `[[gke-master]]` — cluster-level capacity, node pools, and safe maintenance windows.