Lifecycle: draft
Release and Rollback Safety
Purpose and scope
Reduce the blast radius of a change by making release, detection, rollback, and recovery behavior explicit. This skill covers delivery planning and review; it does not replace an organization’s change-approval, incident-response, compliance, or service-ownership process.
Triggers and prerequisites
Trigger when a change will be deployed, changes runtime configuration, schema or data state, feature exposure, dependencies, infrastructure, or release automation. Prerequisites: the change and baseline, deployment topology, owners, health signals, compatibility constraints, rollback authority, data/migration behavior, and an acceptable maintenance window or rollout strategy.
Decision criteria
- A release plan is incomplete without detection, decision ownership, rollback or forward-recovery action, and verification after recovery.
- Prefer progressive exposure and reversible changes when uncertainty or blast radius is high.
- Rollback is not automatically safe when data, schema, queues, external side effects, or irreversible migrations changed; define compatibility and recovery explicitly.
- A green deployment command is not evidence that users received correct behavior or that the system is healthy.
Procedure
- State the intended user and operational outcome, affected components, risk, blast radius, and what must remain unchanged.
- Inventory release prerequisites: artifact identity, configuration, dependencies, migrations, feature flags, permissions, backups, capacity, observability, approvals, and dependency compatibility.
- Define pre-release checks and a release invariant. Include version/configuration verification, migration safety, health criteria, and the evidence required before exposure increases.
- Choose an exposure strategy appropriate to risk: canary, staged rollout, blue/green, feature flag, maintenance window, or another justified path. Define who can pause or abort it.
- Define actionable health signals and thresholds for errors, latency, saturation, correctness, data integrity, security, and user impact. Assign an owner and observation window; avoid thresholds with no response.
- Write the rollback or forward-recovery procedure, including commands or actions, authorization, compatible versions, data/schema handling, queued or external side effects, and how to prevent repeated damage.
- Release to the smallest intended exposure, observe the defined signals, and compare with the baseline. Do not expand exposure because a fixed timer elapsed if the evidence is incomplete.
- If a threshold is crossed, pause exposure and follow the predeclared response. Preserve evidence, communicate scope, contain harm, and do not improvise irreversible actions without authorization.
- After completion or recovery, verify user behavior, data integrity, dependencies, observability, and cleanup. Record what happened, what was learned, residual risk, and the next review trigger.
Examples and counterexamples
Good: A schema change uses an expand/migrate/contract sequence, keeps old and new versions compatible during rollout, monitors errors and data integrity, and defines what happens if the application rollback occurs after the migration.
Bad: Roll back application binaries while assuming the database can always be restored automatically.
Good: A canary has a named owner, bounded exposure, p95 latency and error thresholds, a pause action, and a post-rollout verification period.
Bad: Deploy to all users and decide whether it worked from the absence of an immediate alert.
Failure modes and recovery
If rollback is unsafe or unknown, stop the rollout and design forward recovery or a compatible transition before exposure increases. If health signals are missing, keep the change below trusted release status and add the minimum decision-relevant observability. If an external side effect cannot be undone, contain further effects, record the irreversibility, and escalate. If a release partially succeeds, preserve the exact state and versions before attempting repair.
Validation evidence and provenance
Claims in this skill map to graded findings in docs/research.md:
- Staged rollouts, canaries, and fast rollback limit incident impact (Q10, Moderate): Google SRE release engineering; DORA continuous-delivery research (large multi-year surveys, self-reported).
- Incident discipline during release failures (C2, Moderate): preserved facts, explicit communication, follow-ups.
Source boundary: survey-based evidence links the practice to outcomes but does not prove causation for any specific deployment.
Confidence: medium. Freshness: review when deployment platforms, the research base, or DORA findings change.
Disconfirmation: evidence that staged delivery materially delays incident detection without reducing blast radius would require re-weighting canary stages.
Related skills and conflicts
Related: api-contract-compatibility, observability-and-instrumentation, secure-coding-review, dependency-security-audit, performance-regression-analysis, knowledge-maintenance, and repository-change-verification. This skill does not authorize production changes, bypass approvals, or claim rollback safety without checking data and external side effects.
1---2name: release-and-rollback-safety3description: Plan and review safe software releases with explicit preconditions, staged exposure, health signals, rollback boundaries, and recovery evidence. Use when an AI coding agent changes deployable behavior, migrations, configuration, or release automation.4---56Lifecycle: `draft`78# Release and Rollback Safety910## Purpose and scope1112Reduce the blast radius of a change by making release, detection, rollback, and recovery behavior explicit. This skill covers delivery planning and review; it does not replace an organization’s change-approval, incident-response, compliance, or service-ownership process.1314## Triggers and prerequisites1516Trigger when a change will be deployed, changes runtime configuration, schema or data state, feature exposure, dependencies, infrastructure, or release automation. Prerequisites: the change and baseline, deployment topology, owners, health signals, compatibility constraints, rollback authority, data/migration behavior, and an acceptable maintenance window or rollout strategy.1718## Decision criteria1920- A release plan is incomplete without detection, decision ownership, rollback or forward-recovery action, and verification after recovery.21- Prefer progressive exposure and reversible changes when uncertainty or blast radius is high.22- Rollback is not automatically safe when data, schema, queues, external side effects, or irreversible migrations changed; define compatibility and recovery explicitly.23- A green deployment command is not evidence that users received correct behavior or that the system is healthy.2425## Procedure26271. State the intended user and operational outcome, affected components, risk, blast radius, and what must remain unchanged.282. Inventory release prerequisites: artifact identity, configuration, dependencies, migrations, feature flags, permissions, backups, capacity, observability, approvals, and dependency compatibility.293. Define pre-release checks and a release invariant. Include version/configuration verification, migration safety, health criteria, and the evidence required before exposure increases.304. Choose an exposure strategy appropriate to risk: canary, staged rollout, blue/green, feature flag, maintenance window, or another justified path. Define who can pause or abort it.315. Define actionable health signals and thresholds for errors, latency, saturation, correctness, data integrity, security, and user impact. Assign an owner and observation window; avoid thresholds with no response.326. Write the rollback or forward-recovery procedure, including commands or actions, authorization, compatible versions, data/schema handling, queued or external side effects, and how to prevent repeated damage.337. Release to the smallest intended exposure, observe the defined signals, and compare with the baseline. Do not expand exposure because a fixed timer elapsed if the evidence is incomplete.348. If a threshold is crossed, pause exposure and follow the predeclared response. Preserve evidence, communicate scope, contain harm, and do not improvise irreversible actions without authorization.359. After completion or recovery, verify user behavior, data integrity, dependencies, observability, and cleanup. Record what happened, what was learned, residual risk, and the next review trigger.3637## Examples and counterexamples3839Good: A schema change uses an expand/migrate/contract sequence, keeps old and new versions compatible during rollout, monitors errors and data integrity, and defines what happens if the application rollback occurs after the migration.4041Bad: Roll back application binaries while assuming the database can always be restored automatically.4243Good: A canary has a named owner, bounded exposure, p95 latency and error thresholds, a pause action, and a post-rollout verification period.4445Bad: Deploy to all users and decide whether it worked from the absence of an immediate alert.4647## Failure modes and recovery4849If rollback is unsafe or unknown, stop the rollout and design forward recovery or a compatible transition before exposure increases. If health signals are missing, keep the change below trusted release status and add the minimum decision-relevant observability. If an external side effect cannot be undone, contain further effects, record the irreversibility, and escalate. If a release partially succeeds, preserve the exact state and versions before attempting repair.5051## Validation evidence and provenance52Claims in this skill map to graded findings in [`docs/research.md`](../../docs/research.md):5354- Staged rollouts, canaries, and fast rollback limit incident impact (Q10, Moderate): Google SRE release engineering; DORA continuous-delivery research (large multi-year surveys, self-reported).55- Incident discipline during release failures (C2, Moderate): preserved facts, explicit communication, follow-ups.5657Source boundary: survey-based evidence links the practice to outcomes but does not prove causation for any specific deployment.5859Confidence: medium. Freshness: review when deployment platforms, the research base, or DORA findings change.6061Disconfirmation: evidence that staged delivery materially delays incident detection without reducing blast radius would require re-weighting canary stages.62## Related skills and conflicts6364Related: `api-contract-compatibility`, `observability-and-instrumentation`, `secure-coding-review`, `dependency-security-audit`, `performance-regression-analysis`, `knowledge-maintenance`, and `repository-change-verification`. This skill does not authorize production changes, bypass approvals, or claim rollback safety without checking data and external side effects.
ChloeVPin