Migration Engineering

Migration Engineering

Plan and execute safe migrations across system boundaries. A migration is any change that moves data, schemas, interfaces, infrastructure, or services from a current state to a target state while preserving correctness, availability, and recoverability during the transition.

This skill owns the cross-system migration method — compatibility design, staging, reconciliation, cutover, recovery, and deprecation. It does not own the implementation details of any single technology or subsystem; those belong to specialist skills.

When to use

Load this skill when the task involves:

When not to use

• Single-technology quick fixes — if the change is confined to one system with no compatibility window, no dual-running, and no cross-system coordination, use the relevant specialist skill directly (e.g., data-engineering for a simple DDL change, api-design-and-evolution for a single-endpoint deprecation).
• Tool-specific how-to guides — this skill provides the method, not vendor-specific instructions. It does not prescribe one migration technology, one database engine, one API gateway, or one infrastructure platform.
• Migrations without a system boundary — in-place refactors, code rewrites that don't cross a data or interface boundary, or single-service configuration changes are not migration-engineering scope.
• Guaranteeing rollback — this skill does not claim rollback is always possible. Some migrations include steps that are irreversible; the method requires identifying those steps explicitly and planning acceptance, communication, and contingency rather than implying a false safety net.

Migration type classification

Migrations differ materially in their compatibility, correctness, and recovery characteristics. Classify the migration before selecting patterns.

Schema migration

A change to a database schema, message format, or serialization contract. Compatibility: forward compatibility (old readers tolerate new writers) and backward compatibility (new readers tolerate old writers) are the central design constraints. Correctness: verified by dual-reading or shadow-traffic comparison — the new schema must produce equivalent results for the same input. Rollback: possible if the schema change is purely additive (expand phase); destructive changes (drop column, rename, change type) require a multi-step expand/contract sequence with a compatibility window where both old and new schemas coexist before the old is removed.

Data migration

Movement or transformation of data between stores, representations, or ownership boundaries. Compatibility: the old and new data representations must coexist during the transition; consumers may read from either or both. The backfill strategy (full, incremental, or streaming) determines how long the dual-read window lasts. Correctness: requires reconciliation — a record-level or aggregate comparison between source and target to verify completeness and accuracy before cutover. Rollback: depends on whether the old store remains writable and current during the transition. If the old store is kept in sync (dual-write), rollback is reversing the cutover. If the old store was dropped or made read-only, rollback requires restore from backup.

API migration

A change to the contract between a provider and its consumers — versioning, protocol, schema, or endpoint topology. Compatibility: defined by the provider's compatibility policy (e.g., "additive changes are backward-compatible; removals require a deprecation window"). The compatibility window is measured in consumer migration time — how long consumers need to move from the old interface to the new one. Correctness: verified by consumer-side testing, shadow-traffic replay, and error-rate comparison between old and new interfaces. Rollback: the old interface must remain available and supported throughout the deprecation window; rolling back means reverting the deprecation notice and keeping the old interface live. Once the old interface is removed, rollback requires deploying it again — a restore or redeploy path, not a simple reversal.

Infrastructure and service migration

Moving workloads, services, or infrastructure between environments, platforms, or ownership domains. Compatibility: network, identity, and data-plane continuity must be maintained. DNS, certificates, service discovery, and security boundaries are the primary compatibility surface. Correctness: verified by traffic shifting, canary deployment, and service-level objective (SLO) monitoring during the transition. Rollback: depends on the migration topology. Lift-and-shift with the old environment preserved is reversible; in-place replacement without a preserved old environment may be irreversible or require a full redeploy (roll-forward/restore).

Core workflow

1. Classify and scope the migration

Determine which migration type(s) apply. A real-world migration often combines types — for example, a service extraction includes both a data migration and an API migration. Document:

• The current state, target state, and system boundary being crossed.
• The migration type(s) and their compatibility requirements.
• Which systems, teams, and consumers are affected.

2. Design the expand/contract sequence

The expand/contract pattern is the foundational safe-migration primitive:

1. Expand — add the new interface, schema, or system while the old one continues to serve. Both old and new coexist. This phase is backward-compatible: existing consumers are unaffected.
2. Compatibility window — a defined period (duration or condition) during which both old and new are available. Consumers, replicas, and dependent systems are migrated to the new interface during this window. The window must have an explicit end condition — a date, a metric threshold (e.g., "zero traffic on old endpoint for 7 days"), or an event (e.g., "all registered consumers confirmed migration").
3. Dual-running or parallel operation — for data and service migrations, both systems operate concurrently. Writes may be dual-written; reads may be dual-read with comparison. The dual-running period provides the evidence needed for the cutover decision.
4. Contract — remove the old interface, schema, or system after the compatibility window closes and verification confirms the new system is correct and complete. The contract phase may include data cleanup, code removal, and decommissioning.

Not every migration uses all four phases. A simple additive schema change may use only expand (phase 1) — the old schema keeps working, the new column is added, and no contract phase is needed. A complex service extraction uses all four.

3. Plan the backfill and reconciliation

For data migrations, design the backfill strategy:

• Full backfill — copy all existing data from source to target in a single or batched operation before enabling dual-writes.
• Incremental backfill — copy data in pages or segments; useful for large datasets where a full backfill would exceed the available window.
• Streaming backfill — capture changes from the source via change-data- capture (CDC) or event log and apply them to the target continuously.

Reconciliation — how source and target are verified to match:

Reconciliation runs continuously during the compatibility window and must pass before cutover. A reconciliation failure is a stop condition — the cutover must not proceed.

4. Design the cutover

The cutover is the point where the new system becomes the source of truth. Design for:

• Cutover procedure — the exact sequence of operations, automated where possible, with pre-conditions and post-conditions.
• Cutover window — the expected duration, the acceptable downtime (if any), and the communication plan.
• Interruption points — where the cutover can be paused or reversed. A cutover that has no interruption points is a risk to flag explicitly.
• Observability during cutover — what metrics, logs, and alerts confirm the cutover is proceeding correctly, and what signals trigger abort.

5. Define recovery paths

Every migration step has a recovery classification. Use exactly these four categories — never conflate them:

Irreversible steps require explicit acknowledgment before execution. The migration plan must distinguish "we have chosen not to build a reversal path" from "reversal is physically impossible." Both require acceptance, communication, and contingency — never silent assumption.

6. Plan deprecation and cleanup

After cutover is complete and verified:

• Deprecation window — how long the old system remains available in read-only or degraded mode before removal.
• Consumer migration tracking — which consumers still depend on the old interface, and when they are expected to migrate.
• Cleanup — removal of old schemas, code paths, feature flags, configuration, credentials, and infrastructure.
• Communication — notifications to consumers, stakeholders, and operators at each stage: compatibility window opens, cutover scheduled, cutover complete, deprecation window closing, old system removed.

7. Verify and close

Before declaring the migration complete:

• Correctness evidence — reconciliation reports, consumer verification, error-rate comparisons, SLO compliance data.
• Observability confirmation — migration-specific dashboards and alerts show the expected post-migration steady state.
• Recovery verification — rollback, roll-forward, or restore procedures were tested (where applicable); irreversible steps were acknowledged.
• Owner sign-off — the accountable owner for each phase confirms completion.

Structured planning fields

Every migration plan must address these fields. They may appear as checklist items, template fields, table columns, or labeled section headers — not only as prose.

Reconciliation

Correctness evidence

Observability

Customer impact

Ownership

Loading guide

Load references and templates on demand — do not load everything at once.

Specialist routing

Migration engineering composes domain specialists — it never duplicates their methodology. Route implementation details to the skill that owns the subsystem.

Routing to same-wave and future skills

Migration evidence — reconciliation reports, cutover records, recovery-path classifications, and deprecation tracking — feeds production-readiness assessments. The production-readiness skill consumes migration plans as evidence that a service is ready for production operation.

The production-excellence bundle composes migration-engineering alongside production-readiness, resilience-and-recovery, capacity-and-cost-engineering, incident-learning, and privacy-engineering. Migration-engineering contributes the safe-change dimension to the production-excellence lifecycle.

Routing to product-lifecycle skills

When a migration is triggered by a feature retirement or product sunset, coordinate with product-lifecycle-learning for the retirement decision record, deprecation timeline, and customer-treatment plan.