ai-driven-dev Procedure — rollback
Decide in under two minutes.
- Roll back first, understand after.
kubectl rollout undo deploy/<name>. - Announce it in
#incidentswith the deploy SHA, before investigating. - Check the metric that triggered the alarm has come back. If it has not, the deploy was not the cause and the rollback bought nothing — say so.
- Open an issue with the SHA and what was observed. Same day.
When a rollback is not enough
If the release ran a migration, the rollback does not undo it — that is why migrations are additive and spread over two releases. If a migration was not additive, this procedure does not apply and you are in an incident.