Dual Write Migration Pattern
Dual write keeps two data stores in sync by writing to both the old and new system on every mutation. This enables gradual migration with rollback safety.
When to Use This Skill
| Use this skill when... |
Use shadow-mode instead when... |
| Migrating between databases or schemas |
Validating read-path behavior under real traffic |
| Switching storage backends (SQL to NoSQL, etc.) |
Testing a new service without writing to it |
| Need both systems to stay authoritative during transition |
Only need to compare responses, not persist data |
| Planning zero-downtime data migrations |
Mirroring traffic to a staging environment |
| Reviewing code that writes to multiple data stores |
Evaluating performance of a replacement system |
Core Concepts
Migration Phases
| Phase |
Primary reads |
Primary writes |
Secondary writes |
Duration |
| 1. Prepare |
Old |
Old |
None |
Setup |
| 2. Dual write |
Old |
Old + New |
New (async or sync) |
Migration window |
| 3. Backfill |
Old |
Old + New |
New |
Until parity |
| 4. Shadow read |
Old + New (compare) |
Old + New |
New |
Validation |
| 5. Cutover |
New |
New |
Old (optional) |
Transition |
| 6. Cleanup |
New |
New |
None |
Final |
Write Strategies
| Strategy |
Consistency |
Latency impact |
Failure mode |
| Synchronous |
Strong |
Higher (2x write) |
Fail if either store fails |
| Async secondary |
Eventual |
Minimal |
Secondary may lag |
| Outbox pattern |
Eventual |
Minimal |
Requires message broker |
| Change data capture |
Eventual |
None (DB-level) |
Requires CDC infrastructure |
Implementation Architecture
Synchronous Dual Write
Client Request
│
▼
┌─────────────┐
│ Application │
│ Layer │
└──────┬──────┘
│ write(data)
▼
┌─────────────┐
│ Dual Write │
│ Adapter │
├──────┬──────┤
│ │ │
▼ │ ▼
Old DB │ New DB
│
Compare on
read (optional)
Key Components
| Component |
Responsibility |
| Write adapter |
Routes writes to both stores, handles failures |
| Read comparator |
Reads from both, logs discrepancies, returns primary |
| Backfill job |
Copies historical data from old to new store |
| Reconciliation |
Detects and resolves drift between stores |
| Feature flags |
Controls which phase is active per entity/tenant |
Implementation Patterns
Write Adapter Pattern
The write adapter wraps both stores behind a single interface:
- Accept the write request
- Write to the primary (old) store first
- Write to the secondary (new) store
- If secondary fails: log the failure, enqueue for retry, do not fail the request
- Return the primary store's result to the caller
Read Comparison Pattern
During the shadow read phase:
- Read from the primary (old) store — this is the authoritative response
- Read from the secondary (new) store asynchronously
- Compare results field by field
- Log discrepancies with context (entity ID, field, old value, new value)
- Return the primary store's result
- Track comparison metrics (match rate, common divergence fields)
Backfill Strategy
- Snapshot the old store at a known point in time
- Begin dual writes for all new mutations
- Copy historical records in batches (oldest first or by priority)
- Track backfill progress per entity type
- Reconcile records modified during backfill (dual write wins over backfill)
Failure Handling
| Failure scenario |
Response |
Recovery |
| Secondary write fails |
Log, continue, enqueue retry |
Async retry with backoff |
| Primary write fails |
Fail the request (do not write to secondary) |
Standard error handling |
| Both fail |
Fail the request |
Standard error handling |
| Secondary write timeout |
Log, continue |
Async verification and repair |
| Inconsistency detected |
Log with full context |
Manual or automated reconciliation |
Consistency Guarantees
- Primary store is always the source of truth until cutover
- Secondary store may lag during async dual write
- Reconciliation jobs detect and repair drift
- Cutover only happens when match rate reaches threshold (e.g., 99.9%)
Cutover Decision Criteria
| Metric |
Threshold |
How to measure |
| Read comparison match rate |
> 99.9% |
Shadow read comparison logs |
| Backfill completion |
100% |
Backfill progress tracker |
| Secondary write success rate |
> 99.95% |
Write adapter metrics |
| P99 latency impact |
< 20% increase |
Application metrics |
| Reconciliation gap |
0 unresolved |
Reconciliation job output |
Common Pitfalls
| Pitfall |
Mitigation |
| Ordering issues between stores |
Use idempotent writes, include version/timestamp |
| Transaction boundaries differ |
Design writes to be independently valid |
| Schema mismatch between stores |
Map fields explicitly, handle nullability differences |
| Backfill conflicts with live writes |
Live dual writes take precedence over backfill |
| Performance degradation |
Start with async secondary writes |
| Partial failures leave inconsistency |
Reconciliation job as safety net |
| Forgetting to dual-write in all code paths |
Centralize through write adapter, audit call sites |
Rollback Plan
| Phase |
Rollback action |
Data impact |
| Dual write |
Stop writing to new store |
No data loss |
| Shadow read |
Stop comparing reads |
No data loss |
| Cutover (reads) |
Switch reads back to old |
No data loss if still dual-writing |
| Cutover (writes) |
Reverse write order |
May need reconciliation |
| Cleanup |
Cannot rollback |
Old store decommissioned |
Monitoring Checklist
Agentic Optimizations
| Context |
Approach |
| Code review |
Check that all write paths go through the dual-write adapter |
| Architecture review |
Verify failure handling, rollback plan, and cutover criteria |
| Implementation |
Start with write adapter + async secondary, add comparison later |
| Testing |
Simulate secondary failures, verify primary is unaffected |
Quick Reference
| Term |
Definition |
| Primary store |
The authoritative data store (old system during migration) |
| Secondary store |
The new data store being migrated to |
| Backfill |
Copying historical data from primary to secondary |
| Reconciliation |
Detecting and repairing differences between stores |
| Cutover |
Switching the primary designation from old to new |
| Match rate |
Percentage of shadow reads that return identical results |
| Write adapter |
Abstraction layer that routes writes to both stores |
1---2name: dual-write3description: Dual-write pattern for safe data store transitions. Use when planning DB migrations, switching storage backends, or reviewing code writing to multiple systems simultaneously.4---5
6# Dual Write Migration Pattern
7
8Dual write keeps two data stores in sync by writing to both the old and new system on every mutation. This enables gradual migration with rollback safety.
9
10## When to Use This Skill
11
12| Use this skill when... | Use shadow-mode instead when... |
13|------------------------|--------------------------------|
14| Migrating between databases or schemas | Validating read-path behavior under real traffic |
15| Switching storage backends (SQL to NoSQL, etc.) | Testing a new service without writing to it |
16| Need both systems to stay authoritative during transition | Only need to compare responses, not persist data |
17| Planning zero-downtime data migrations | Mirroring traffic to a staging environment |
18| Reviewing code that writes to multiple data stores | Evaluating performance of a replacement system |
19
20## Core Concepts
21
22### Migration Phases
23
24| Phase | Primary reads | Primary writes | Secondary writes | Duration |
25|-------|--------------|----------------|------------------|----------|
26| 1. Prepare | Old | Old | None | Setup |
27| 2. Dual write | Old | Old + New | New (async or sync) | Migration window |
28| 3. Backfill | Old | Old + New | New | Until parity |
29| 4. Shadow read | Old + New (compare) | Old + New | New | Validation |
30| 5. Cutover | New | New | Old (optional) | Transition |
31| 6. Cleanup | New | New | None | Final |
32
33### Write Strategies
34
35| Strategy | Consistency | Latency impact | Failure mode |
36|----------|------------|----------------|--------------|
37| Synchronous | Strong | Higher (2x write) | Fail if either store fails |
38| Async secondary | Eventual | Minimal | Secondary may lag |
39| Outbox pattern | Eventual | Minimal | Requires message broker |
40| Change data capture | Eventual | None (DB-level) | Requires CDC infrastructure |
41
42## Implementation Architecture
43
44### Synchronous Dual Write
45
46```
47Client Request
48 │
49 ▼
50┌─────────────┐
51│ Application │
52│ Layer │
53└──────┬──────┘
54 │ write(data)
55 ▼
56┌─────────────┐
57│ Dual Write │
58│ Adapter │
59├──────┬──────┤
60│ │ │
61▼ │ ▼
62Old DB │ New DB
63 │
64 Compare on
65 read (optional)
66```
67
68### Key Components
69
70| Component | Responsibility |
71|-----------|---------------|
72| Write adapter | Routes writes to both stores, handles failures |
73| Read comparator | Reads from both, logs discrepancies, returns primary |
74| Backfill job | Copies historical data from old to new store |
75| Reconciliation | Detects and resolves drift between stores |
76| Feature flags | Controls which phase is active per entity/tenant |
77
78## Implementation Patterns
79
80### Write Adapter Pattern
81
82The write adapter wraps both stores behind a single interface:
83
841. Accept the write request
852. Write to the primary (old) store first
863. Write to the secondary (new) store
874. If secondary fails: log the failure, enqueue for retry, do not fail the request
885. Return the primary store's result to the caller
89
90### Read Comparison Pattern
91
92During the shadow read phase:
93
941. Read from the primary (old) store — this is the authoritative response
952. Read from the secondary (new) store asynchronously
963. Compare results field by field
974. Log discrepancies with context (entity ID, field, old value, new value)
985. Return the primary store's result
996. Track comparison metrics (match rate, common divergence fields)
100
101### Backfill Strategy
102
1031. Snapshot the old store at a known point in time
1042. Begin dual writes for all new mutations
1053. Copy historical records in batches (oldest first or by priority)
1064. Track backfill progress per entity type
1075. Reconcile records modified during backfill (dual write wins over backfill)
108
109## Failure Handling
110
111| Failure scenario | Response | Recovery |
112|-----------------|----------|----------|
113| Secondary write fails | Log, continue, enqueue retry | Async retry with backoff |
114| Primary write fails | Fail the request (do not write to secondary) | Standard error handling |
115| Both fail | Fail the request | Standard error handling |
116| Secondary write timeout | Log, continue | Async verification and repair |
117| Inconsistency detected | Log with full context | Manual or automated reconciliation |
118
119### Consistency Guarantees
120
121- Primary store is always the source of truth until cutover
122- Secondary store may lag during async dual write
123- Reconciliation jobs detect and repair drift
124- Cutover only happens when match rate reaches threshold (e.g., 99.9%)
125
126## Cutover Decision Criteria
127
128| Metric | Threshold | How to measure |
129|--------|-----------|----------------|
130| Read comparison match rate | > 99.9% | Shadow read comparison logs |
131| Backfill completion | 100% | Backfill progress tracker |
132| Secondary write success rate | > 99.95% | Write adapter metrics |
133| P99 latency impact | < 20% increase | Application metrics |
134| Reconciliation gap | 0 unresolved | Reconciliation job output |
135
136## Common Pitfalls
137
138| Pitfall | Mitigation |
139|---------|-----------|
140| Ordering issues between stores | Use idempotent writes, include version/timestamp |
141| Transaction boundaries differ | Design writes to be independently valid |
142| Schema mismatch between stores | Map fields explicitly, handle nullability differences |
143| Backfill conflicts with live writes | Live dual writes take precedence over backfill |
144| Performance degradation | Start with async secondary writes |
145| Partial failures leave inconsistency | Reconciliation job as safety net |
146| Forgetting to dual-write in all code paths | Centralize through write adapter, audit call sites |
147
148## Rollback Plan
149
150| Phase | Rollback action | Data impact |
151|-------|----------------|-------------|
152| Dual write | Stop writing to new store | No data loss |
153| Shadow read | Stop comparing reads | No data loss |
154| Cutover (reads) | Switch reads back to old | No data loss if still dual-writing |
155| Cutover (writes) | Reverse write order | May need reconciliation |
156| Cleanup | Cannot rollback | Old store decommissioned |
157
158## Monitoring Checklist
159
160- [ ] Write success rate per store
161- [ ] Write latency per store (P50, P95, P99)
162- [ ] Read comparison match rate
163- [ ] Backfill progress percentage
164- [ ] Reconciliation queue depth
165- [ ] Error rate by failure type
166- [ ] Feature flag state per tenant/entity
167
168## Agentic Optimizations
169
170| Context | Approach |
171|---------|----------|
172| Code review | Check that all write paths go through the dual-write adapter |
173| Architecture review | Verify failure handling, rollback plan, and cutover criteria |
174| Implementation | Start with write adapter + async secondary, add comparison later |
175| Testing | Simulate secondary failures, verify primary is unaffected |
176
177## Quick Reference
178
179| Term | Definition |
180|------|-----------|
181| Primary store | The authoritative data store (old system during migration) |
182| Secondary store | The new data store being migrated to |
183| Backfill | Copying historical data from primary to secondary |
184| Reconciliation | Detecting and repairing differences between stores |
185| Cutover | Switching the primary designation from old to new |
186| Match rate | Percentage of shadow reads that return identical results |
187| Write adapter | Abstraction layer that routes writes to both stores |
laurigates