Error Handler

Error Handler

Produce a complete error handling design for a system integration. Every error category gets an explicit retry strategy, escalation path, and resolution workflow. The output is the technical specification a developer implements in the integration layer.

Error Taxonomy

Classify every possible error into one of three categories. The category determines the retry strategy.

Category 1: Transient Errors (Retry Eligible)

The error is temporary. The operation will likely succeed if retried.

Retry strategy for transient errors:

Algorithm: Exponential backoff with full jitter

base_delay = 1 second
max_delay = 60 seconds
max_attempts = 5
jitter = random(0, base_delay)

wait_time(attempt) = min(base_delay * 2^attempt + jitter, max_delay)

Attempt 1: 0s (immediate)
Attempt 2: ~2s (base*2 + jitter)
Attempt 3: ~4s (base*4 + jitter)
Attempt 4: ~8s (base*8 + jitter)
Attempt 5: ~16s (base*16 + jitter)
After attempt 5: Send to dead-letter queue

Rate limit (429) special handling: If the response includes a Retry-After header, use that value instead of the exponential backoff calculation. The Retry-After value is authoritative.

Category 2: Permanent Errors (Do Not Retry)

The operation will fail regardless of how many times it is retried. Retrying wastes resources and delays detection.

Duplicate record (409) handling:

On 409 response:
  1. Extract the existing record identifier from the 409 response body
  2. Issue a GET request to fetch the existing record
  3. Compare key fields: if destination record matches source, mark as "already synced" and continue
  4. If destination record differs, issue a PUT/PATCH to update the existing record
  5. If update succeeds: log "409 resolved via update"
  6. If update fails: send to exception queue

Category 3: Business Errors (Route to Exception Queue)

The operation is technically valid but cannot be processed automatically due to a business rule or data quality issue.

Retry Implementation

Retry Pseudocode

async function withRetry<T>(
  operation: () => Promise<T>,
  config: RetryConfig
): Promise<T> {
  let lastError: Error;
  
  for (let attempt = 0; attempt <= config.maxAttempts; attempt++) {
    try {
      return await operation();
    } catch (error) {
      lastError = error;
      const errorCategory = classifyError(error);
      
      if (errorCategory !== ErrorCategory.TRANSIENT) {
        // Non-transient: do not retry
        throw new NonRetryableError(error, errorCategory);
      }
      
      if (attempt === config.maxAttempts) {
        // Exhausted retries
        throw new RetriesExhaustedError(error, attempt);
      }
      
      const delay = calculateBackoff(attempt, config, error);
      logger.warn('Transient error, retrying', { attempt, delay, error: error.message });
      await sleep(delay);
    }
  }
  
  throw lastError!;
}

function calculateBackoff(attempt: number, config: RetryConfig, error: Error): number {
  // Respect Retry-After header if present
  if (error instanceof ApiError && error.retryAfterSeconds) {
    return error.retryAfterSeconds * 1000;
  }
  
  const exponential = config.baseDelayMs * Math.pow(2, attempt);
  const jitter = Math.random() * config.baseDelayMs;
  return Math.min(exponential + jitter, config.maxDelayMs);
}

Error Classification

enum ErrorCategory {
  TRANSIENT = 'TRANSIENT',
  PERMANENT = 'PERMANENT',
  BUSINESS = 'BUSINESS'
}

function classifyError(error: unknown): ErrorCategory {
  if (error instanceof ApiError) {
    if ([429, 503, 504].includes(error.status)) return ErrorCategory.TRANSIENT;
    if (error.status === 500 && error.body?.includes('retry')) return ErrorCategory.TRANSIENT;
    if ([400, 403, 422].includes(error.status)) return ErrorCategory.PERMANENT;
    if (error.status === 409) return ErrorCategory.PERMANENT; // handled separately
    if (error.status === 404) return ErrorCategory.PERMANENT;
  }
  if (error instanceof NetworkError) return ErrorCategory.TRANSIENT;
  if (error instanceof ValidationError) return ErrorCategory.PERMANENT;
  if (error instanceof BusinessRuleError) return ErrorCategory.BUSINESS;
  
  // Unknown errors: treat as transient for safety, but cap at 2 retries
  return ErrorCategory.TRANSIENT;
}

Dead-Letter Queue Design

The DLQ is the landing zone for records that failed all retry attempts. It must support manual review and reprocessing.

DLQ storage: SharePoint list (for small integrations) or Azure Table Storage (for high-volume integrations).

DLQ schema:

PII redaction in DLQ: Before storing the SourcePayload, redact sensitive fields (SSN, account numbers, dates of birth). Replace with [REDACTED]. Store only enough context to identify and reproduce the issue.

DLQ monitoring dashboard (Power BI report page or SharePoint view):

• Open DLQ items by category (Transient-Exhausted / Permanent / Business)
• DLQ items by integration and source system
• DLQ age distribution: how many items are 0-24h, 1-7d, 7-30d, >30d old
• Resolution rate: what % of DLQ items are resolved within 24 hours

Manual Reprocess Workflow

Provide a safe reprocess mechanism for DLQ items:

Reprocess single item:

1. DLQ reviewer reads the DLQ item and the error message
2. Corrects the underlying issue (fixes source data, adds missing reference, updates mapping)
3. Updates the DLQ item Status to "Under Investigation" and AssignedTo their name
4. Calls the reprocess function with forceReprocess: true (bypasses idempotency check for this item)
5. Monitors the processing result
6. If successful: updates DLQ Status to "Resolved", adds resolution note
7. If still failing: escalates to integration team

Reprocess function:

async function reprocessDlqItem(
  dlqItemId: string,
  forceReprocess: boolean = false
): Promise<void> {
  const item = await dlqStore.getItem(dlqItemId);
  if (!item) throw new Error(`DLQ item not found: ${dlqItemId}`);
  
  logger.info('Manual DLQ reprocess initiated', { dlqItemId, forceReprocess });
  
  await integrationQueue.enqueue({
    ...JSON.parse(item.sourcePayload),
    _dlqReprocess: true,
    _forceReprocess: forceReprocess,
    _dlqItemId: dlqItemId
  });
}

Bulk reprocess (for systematic failures fixed by a configuration change):

• Export DLQ items to CSV, filter to the affected error code, resubmit in batches
• Maximum 100 items per bulk reprocess — prevents overwhelming the destination API

Alert Thresholds

Error Log Retention

Output Format

Deliver as:

1. Error taxonomy table (three categories with all error types)
2. Retry strategy specification (algorithm with pseudocode)
3. Error classification function (pseudocode)
4. DLQ schema (full table with all fields)
5. DLQ monitoring dashboard specification
6. Manual reprocess workflow (numbered steps + pseudocode)
7. Alert threshold table
8. Error log retention policy
9. Integration-specific error codes (any error codes unique to the source or destination API, with handling instructions)