diegosouzapw Idempotency Best Practices
This skill provides idempotency concepts, patterns, and implementation strategies applicable across multiple programming languages and frameworks.
Core Concepts
What is Idempotency?
An operation is idempotent if performing it multiple times produces the same result as performing it once.
Mathematical Example:
f(f(x)) = f(x)- Example:
abs(abs(-5)) = abs(-5) = 5
Programming Example:
- GET request: Always returns same result
- DELETE request: Deleting twice has same effect as once
- PUT request: Replacing resource multiple times has same effect
Why Idempotency Matters
- Retry Safety: Network failures can cause retries - operations must handle this
- Distributed Systems: Multiple services may attempt same operation
- User Experience: Users may click "Submit" multiple times
- Reliability: Ensures system consistency under failures
HTTP Methods and Idempotency
Idempotent Methods
GET: Always safe and idempotent
- Reading data doesn't change state
- Can be called multiple times safely
PUT: Idempotent (when used correctly)
- Replacing resource with same data multiple times has same effect
- Must include complete resource representation
DELETE: Idempotent
- Deleting already-deleted resource has same effect (no-op)
Non-Idempotent Methods
POST: Not idempotent
- Creates new resources
- Each call may create different resource
PATCH: Not idempotent (usually)
- Partial updates may have different effects on repeated calls
API Idempotency Patterns
Pattern 1: Idempotency Keys
Client provides unique key for each operation. Server uses key to detect duplicates.
// Client sends request with idempotency key
POST /api/orders
Headers:
Idempotency-Key: abc123-def456-ghi789
// Server implementation
const idempotencyKey = req.headers['idempotency-key'];
if (idempotencyKey) {
// Check if we've seen this key before
const cachedResponse = await cache.get(`idempotency:${idempotencyKey}`);
if (cachedResponse) {
return res.status(cachedResponse.status).json(cachedResponse.body);
}
// Process request
const order = await createOrder(req.body);
// Cache response
await cache.set(`idempotency:${idempotencyKey}`, {
status: 201,
body: order,
}, { ttl: 24 * 60 * 60 }); // 24 hours
return res.status(201).json(order);
}
Pattern 2: Request Deduplication
class IdempotencyHandler {
async handleRequest(
idempotencyKey: string,
handler: () => Promise<any>
): Promise<any> {
// Check if request already processed
const existing = await this.getResponse(idempotencyKey);
if (existing) {
return existing;
}
// Process request
try {
const result = await handler();
await this.storeResponse(idempotencyKey, result);
return result;
} catch (error) {
// Don't cache errors - allow retry
throw error;
}
}
private async getResponse(key: string) {
return cache.get(`idempotency:${key}`);
}
private async storeResponse(key: string, response: any) {
await cache.set(`idempotency:${key}`, response, { ttl: 86400 });
}
}
Database Idempotency Patterns
Pattern 1: UPSERT Operations
-- PostgreSQL
INSERT INTO orders (id, user_id, total, status)
VALUES ($1, $2, $3, 'pending')
ON CONFLICT (id) DO UPDATE SET
total = EXCLUDED.total,
updated_at = NOW()
RETURNING *;
// MongoDB
await db.orders.findOneAndUpdate(
{ id: orderId },
{ $set: { total, status: 'pending' } },
{ upsert: true, new: true }
);
Pattern 2: Conditional Updates
// Only update if version matches (optimistic locking)
async function updateOrder(orderId: string, data: Partial<Order>, expectedVersion: number) {
const result = await db.orders.updateOne(
{
id: orderId,
version: expectedVersion, // Only update if version matches
},
{
$set: { ...data, version: expectedVersion + 1 },
}
);
if (result.matchedCount === 0) {
throw new Error('Order was modified by another operation');
}
return result;
}
Pattern 3: Unique Constraints
-- Prevent duplicate operations using unique constraint
CREATE UNIQUE INDEX idx_order_user_unique
ON orders(user_id, order_number);
-- Insert will fail if duplicate exists
INSERT INTO orders (user_id, order_number, total)
VALUES ($1, $2, $3);
-- If duplicate, operation is idempotent (no change)
Implementation Strategies
Strategy 1: Versioning
Use version numbers or timestamps to detect changes.
interface Resource {
id: string;
version: number;
data: any;
}
async function updateResource(id: string, data: any, expectedVersion: number) {
const resource = await getResource(id);
if (resource.version !== expectedVersion) {
throw new Error('Resource was modified');
}
return updateResourceWithVersion(id, data, expectedVersion + 1);
}
Strategy 2: State Machines
Use state machines to ensure operations only happen in valid states.
enum OrderStatus {
PENDING = 'pending',
PROCESSING = 'processing',
COMPLETED = 'completed',
CANCELLED = 'cancelled',
}
async function processOrder(orderId: string) {
const order = await getOrder(orderId);
// Only process if in pending state
if (order.status !== OrderStatus.PENDING) {
return order; // Idempotent: return current state
}
// Update to processing
await updateOrderStatus(orderId, OrderStatus.PROCESSING);
try {
await fulfillOrder(order);
await updateOrderStatus(orderId, OrderStatus.COMPLETED);
} catch (error) {
await updateOrderStatus(orderId, OrderStatus.CANCELLED);
throw error;
}
}
Strategy 3: Tokens/Request IDs
Generate unique tokens for each operation attempt.
async function createPayment(paymentData: PaymentData, requestId: string) {
// Check if this request was already processed
const existing = await db.payments.findOne({ requestId });
if (existing) {
return existing; // Return existing payment (idempotent)
}
// Create new payment with request ID
const payment = await db.payments.create({
...paymentData,
requestId,
status: 'pending',
});
return payment;
}
Best Practices
✅ DO:
- Always Return Same Result: Same input → same output
- Use Idempotency Keys: For POST/PATCH operations
- Check Before Creating: Verify resource doesn't exist
- Use UPSERT: For create-or-update operations
- Log Idempotent Operations: Track for debugging
- Handle Retries: Design for automatic retries
- Version Resources: Use versioning for updates
❌ DON'T:
- Increment Counters: Without checking current value
- Create Multiple Resources: From same operation
- Side Effects: Uncontrolled side effects
- Ignore Duplicates: Always check for existing operations
- Cache Errors: Don't cache error responses as idempotent
Common Patterns
Pattern: Idempotent Create
async function createOrder(orderData: OrderData, idempotencyKey: string) {
// Check if order with this key already exists
const existing = await db.orders.findOne({ idempotencyKey });
if (existing) {
return existing; // Return existing order
}
// Create new order
const order = await db.orders.create({
...orderData,
idempotencyKey,
});
return order;
}
Pattern: Idempotent Update
async function updateOrder(orderId: string, updates: Partial<Order>) {
// Get current order
const order = await db.orders.findById(orderId);
// Check if update would change anything
const hasChanges = Object.keys(updates).some(
key => order[key] !== updates[key]
);
if (!hasChanges) {
return order; // No changes, return current state (idempotent)
}
// Apply updates
return db.orders.update(orderId, updates);
}
Key Principles
- Same Input, Same Output: Idempotent operations always return same result
- Safe to Retry: Operations can be safely retried
- Check Before Act: Verify state before performing operation
- Use Idempotency Keys: For non-idempotent HTTP methods
- Version Resources: Track changes to detect conflicts
- State Machines: Use states to control operation flow
- Log Operations: Track idempotent operations for debugging