Evolve a Schema Safely
Treat schema evolution as a distributed state transition, not a single DDL or file-format edit.
Protocol
- Identify producers, consumers, storage, ownership, volume, update frequency, deployment topology, and the maximum period old/new versions can coexist.
- State current and target invariants. Profile real data safely for nulls, duplicates, malformed values, cardinality, skew, and referential gaps; do not infer cleanliness from application types.
- Classify compatibility for old reader/new writer, new reader/old writer, rollback, replay, and delayed consumers. Include caches, replicas, queues, exports, analytics, and offline jobs when touched.
- Prefer expand → migrate → verify → contract:
- expand with additive, backward-compatible shape;
- deploy tolerant readers and controlled dual/single writers;
- backfill in bounded, resumable, idempotent batches;
- verify convergence and constraint readiness;
- contract only after old versions and lagging data are gone.
- Define transaction scope, locking, load limits, throttling, checkpointing, retry semantics, and behavior under partial failure.
- Make rollback truthful. Distinguish code rollback, traffic rollback, forward repair, and data reversal. Destructive transformations may not be reversible without a preserved source or compensating artifact.
- Rehearse on representative disposable data. Verify repeated execution, interruption/resume, mixed versions, boundary values, and rollback/repair.
- Gate each phase with measurable entry/exit criteria and telemetry. Remove compatibility code only after evidence shows it is unused.
Guardrails
- Never run a production migration, backfill, destructive statement, or live data repair without explicit authority.
- Do not add a non-null/unique/foreign-key constraint before proving existing and concurrent writes satisfy it.
- Do not use unbounded updates or assume transactional DDL/rollback semantics.
- Avoid dual writes without defined ordering, failure reconciliation, and a source of truth.
- Treat event and serialized schemas as durable APIs; historical data and delayed consumers are part of compatibility.
Output
Use references/migration-plan.md. Lead with compatibility strategy and the
irreversible point, then phases, gates, recovery, and proof.