HTTP Client

External API Client Rules

1. Timeout Configuration Principles

Recommended Defaults

Timeout Types Explained

Timeout Rules

• Always set explicit timeouts — never rely on defaults (which may be infinite)
• Set read timeout based on the slowest expected response from upstream
• Set connect timeout shorter than read timeout (connection should be fast; 1-5s typical)
• Write timeout matters primarily for large request bodies (file uploads, bulk data)
• Use shorter timeouts for non-critical calls
• Log timeout values at startup for debugging
• For downstream SLA compliance: set total timeout (connect + read) below your own SLA budget minus processing time

2. Error Handling Strategy

Error Classification

Error Handling Rules

• Never let raw HTTP client exceptions propagate to callers
• Wrap in domain-specific exceptions (e.g., PaymentApiException)
• Log response status and body on errors (but mask sensitive data)
• Distinguish between retryable (network, 503) and non-retryable (400, 404) errors

3. Retry Strategy

Retry Decision Table

Exponential Backoff with Jitter

Exponential backoff prevents overwhelming a recovering service. Jitter prevents synchronized retry storms (thundering herd).

Base formula:
  delay = min(base * 2^attempt, max_delay)

Full jitter (recommended — distributes retries evenly):
  delay = random(0, min(base * 2^attempt, max_delay))

Equal jitter (balanced between spread and minimum wait):
  temp = min(base * 2^attempt, max_delay)
  delay = temp/2 + random(0, temp/2)

Example with base=1s, max=30s:
  Attempt 1: random(0, 2s)
  Attempt 2: random(0, 4s)
  Attempt 3: random(0, 8s)

Reference: AWS Architecture Blog — Exponential Backoff and Jitter

Retry Principles

• Always use exponential backoff with jitter to avoid thundering herd
• Prefer full jitter over equal jitter or decorrelated jitter for most use cases
• Set a maximum retry count — never retry indefinitely
• Set a retry budget (e.g., max 20% of requests can be retries) to prevent retry amplification across service layers
• Do not retry inside a database transaction — retry at the outermost layer
• Do not retry non-idempotent requests without an idempotency key
• Ensure retried requests are safe: check that the upstream API is idempotent or use idempotency keys

4. Circuit Breaker Pattern

When to Use

• External APIs with unpredictable latency or availability
• Non-critical dependencies where degraded operation is acceptable
• High-throughput paths where cascading failures are a risk

Key Parameters

5. Response Mapping Principles

DTO Separation

• Never expose external API DTOs to internal service layers
• Map external responses to domain models at the client boundary
• Handle missing/null fields defensively in external DTOs

Mapping Example

External API Response (their naming)     Domain Model (our naming)
─────────────────────────────────────    ──────────────────────────
user_id: String                    →     id: String
full_name: String                  →     name: String
email_address: String              →     email: String

6. Connection Pooling

Why Connection Pooling Matters

Creating a new TCP connection (and TLS handshake) for every request adds 50-200ms of latency. Connection pools maintain reusable connections to amortize this cost.

Pool Configuration Parameters

Connection Pool Pitfalls

Sizing Guidelines

Pool size per host = peak_concurrent_requests_to_that_host * 1.5

Example:
  - Your service handles 1000 req/s
  - 30% of requests call Payment API (300 req/s)
  - Payment API avg response time: 200ms
  - Concurrent connections needed: 300 * 0.2 = 60
  - Pool size: 60 * 1.5 = 90

7. Anti-Patterns

• No timeout configuration (risk of thread exhaustion)
• Retrying non-idempotent requests (POST without idempotency key)
• Logging full response bodies in production (performance + data leak)
• Sharing a single HTTP client instance for unrelated APIs with different latency profiles
• Calling external APIs inside database transactions
• Ignoring rate limit headers from upstream APIs
• No circuit breaker on critical external dependencies
• Not setting connection pool idle/max lifetime (stale connections, DNS caching issues)
• Retry amplification across service layers (service A retries 3x, calls service B which retries 3x = 9 attempts)

8. Related Skills

• error-handling: HTTP error response handling strategies
• observability: HTTP client metrics collection and monitoring
• security: API authentication, TLS, header security
• caching: HTTP response caching strategies

Additional References

• For circuit breaker, retry, timeout, and bulkhead resilience patterns, see references/resilience-patterns.md
• For Spring Boot implementation patterns (RestClient, Spring Retry, Resilience4j), see spring-framework skill — references/http-client.md