API Gateway Design

API Gateway and BFF Pattern

What an API Gateway Does

An API gateway is the single entry point for all client traffic. It handles cross-cutting concerns centrally so individual services don't have to.

CLIENTS                     API GATEWAY                    INTERNAL SERVICES
-------                     -----------                    -----------------
                         +------------------+
Web Browser  ─────────>  | Authentication   |  ──────>  Order Service    :8081
                         | Rate Limiting    |
Mobile App   ─────────>  | Routing          |  ──────>  Product Service  :8082
                         | SSL Termination  |
IoT Device   ─────────>  | Request Transform|  ──────>  Payment Service  :8083
                         | Circuit Breaking |
Partner API  ─────────>  | Caching          |  ──────>  User Service     :8084
                         | Observability    |
                         +------------------+
                         HTTPS (external)       HTTP/gRPC (internal)

Gateway Responsibilities

1. Routing

# Kong: route /api/v1/orders → order-service
_format_version: "3.0"
services:
- name: order-service
  url: http://order-service:8081
  routes:
  - name: orders-route
    paths: ["/api/v1/orders"]
    methods: ["GET", "POST"]
    strip_path: false

- name: product-service
  url: http://product-service:8082
  routes:
  - name: products-route
    paths: ["/api/v1/products"]
    methods: ["GET"]

2. Authentication — Verify Before Forwarding

Never let unauthenticated requests reach internal services. Validate at the gateway; forward claims as headers.

# Kong JWT plugin: validate at gateway, forward claims downstream
plugins:
- name: jwt
  service: order-service
  config:
    key_claim_name: kid
    claims_to_verify:
    - exp
    - nbf
    header_names:
    - Authorization

# Custom auth middleware (nginx + lua / Kong plugin pattern)
def validate_and_forward(request):
    token = request.headers.get("Authorization", "").removeprefix("Bearer ")
    
    try:
        payload = jwt.decode(
            token,
            public_key,
            algorithms=["RS256"],
            audience="api.example.com",
            issuer="auth.example.com"
        )
    except jwt.ExpiredSignatureError:
        return Response(401, {"error": "token_expired"})
    except jwt.InvalidTokenError:
        return Response(401, {"error": "invalid_token"})
    
    # Forward claims as headers — services trust these (internal network)
    request.headers["X-User-Id"] = payload["sub"]
    request.headers["X-User-Role"] = payload.get("role", "user")
    request.headers["X-Tenant-Id"] = payload.get("tenant_id", "")
    
    return forward_to_upstream(request)

3. Rate Limiting

RATE LIMITING ALGORITHMS

Fixed Window          Sliding Window        Token Bucket          Leaky Bucket
------------          --------------        ------------          ------------
|  window 1  |        continuous window     tokens at rate R      queue at rate R
| count: 95  |        precise, no burst     burst up to B         smooth output
| [burst!]   |        more memory           most common           no burst allowed
|  window 2  |        Redis sorted set      Redis counter         queue depth = B
| count: 105 |        per user              per user per endpoint
| → 429!     |

Recommendation: Token Bucket for most APIs

# Token bucket rate limiting with Redis
import redis
import time

r = redis.Redis()

def check_rate_limit(client_id: str, endpoint: str, 
                     rate: int = 100, capacity: int = 200) -> tuple[bool, dict]:
    """
    rate: tokens added per second
    capacity: max burst size (bucket capacity)
    Returns: (allowed, headers)
    """
    key = f"ratelimit:{client_id}:{endpoint}"
    now = time.time()
    
    pipe = r.pipeline()
    pipe.hgetall(key)
    pipe.expire(key, 3600)
    result, _ = pipe.execute()
    
    if result:
        tokens = float(result[b'tokens'])
        last_refill = float(result[b'last_refill'])
        # Add tokens since last refill
        elapsed = now - last_refill
        tokens = min(capacity, tokens + elapsed * rate)
    else:
        tokens = float(capacity)
        last_refill = now
    
    if tokens >= 1.0:
        tokens -= 1.0
        allowed = True
    else:
        allowed = False
    
    pipe = r.pipeline()
    pipe.hset(key, mapping={'tokens': tokens, 'last_refill': now})
    pipe.expire(key, 3600)
    pipe.execute()
    
    retry_after = int((1.0 - tokens) / rate) if not allowed else 0
    headers = {
        "X-RateLimit-Limit": str(rate),
        "X-RateLimit-Remaining": str(int(tokens)),
        "X-RateLimit-Reset": str(int(now + (capacity - tokens) / rate)),
        **({"Retry-After": str(retry_after)} if not allowed else {})
    }
    return allowed, headers

# Kong rate-limiting plugin: tier-based limits
plugins:
- name: rate-limiting
  config:
    second: null
    minute: 60        # 60 req/min for anonymous
    hour: 1000
    policy: redis
    redis_host: redis
    redis_port: 6379

# Per consumer (API key tier)
consumers:
- username: partner-acme
  plugins:
  - name: rate-limiting
    config:
      minute: 600     # 10x higher for partners
      hour: 50000

4. SSL Termination

Client ─── HTTPS (TLS 1.3) ──> Gateway ─── HTTP ──> Service A
                                         └── HTTP ──> Service B
                                         └── gRPC  ──> Service C
              cert lives at gateway;
              internal traffic is plain (within private VPC)
              or mTLS if service mesh is used

5. Circuit Breaking at Gateway

# Kong circuit breaker (Upstream health checks)
upstreams:
- name: order-service-upstream
  healthchecks:
    active:
      http_path: /health/ready
      healthy:
        interval: 10
        successes: 2
      unhealthy:
        interval: 5
        http_failures: 3
        timeouts: 3
    passive:
      healthy:
        successes: 5
      unhealthy:
        http_failures: 5
        http_statuses: [500, 502, 503, 504]
        timeouts: 3
  targets:
  - target: order-service-1:8081
    weight: 100
  - target: order-service-2:8081
    weight: 100

BFF (Backend for Frontend) Pattern

Problem: One Generic API Serves All Clients Badly

WRONG: Single generic API
        API
         |
    +---------+
    |         |         Web gets TOO MUCH data (over-fetching)
   Web      Mobile      Mobile makes TOO MANY requests (under-fetching)
                        TV app needs completely different data shape

RIGHT: BFF per client type
        Web BFF          Mobile BFF        TV BFF
            |                 |               |
    +-------+-------+---------+-----+---------+
    |               |               |         |
 Order Svc    Product Svc    User Svc    Catalog Svc
 (internal)   (internal)    (internal)  (internal)

BFF Responsibilities Per Client

WEB BFF                         MOBILE BFF                    TV BFF
-------                         ----------                    ------
Complex aggregations             Minimal payloads              Lean content lists
Full product data                Compressed images only        No user account ops
Session-based auth (cookies)     JWT token auth                Device-registered tokens
Server-side rendering support    Offline-first endpoints       Simple navigation-only API
Admin features                   Push notification hooks       Autoplay/continue-watching
Rich pagination                  Cursor-based pagination       Simple next/prev
WebSockets for real-time         Long polling fallback         Polling (smart TV limits)

BFF Implementation (Spring Boot)

// Mobile BFF: aggregate order + product data into minimal payload
@RestController
@RequestMapping("/mobile/v1")
public class MobileOrderBff {

    private final OrderServiceClient orderClient;
    private final ProductServiceClient productClient;
    private final UserServiceClient userClient;

    // Aggregate: one call returns everything mobile checkout screen needs
    @GetMapping("/orders/{orderId}/checkout-summary")
    public Mono<CheckoutSummary> getCheckoutSummary(
            @PathVariable String orderId,
            @RequestHeader("X-User-Id") String userId) {

        // Fan-out: call 3 services concurrently
        Mono<Order> orderMono = orderClient.getOrder(orderId);
        Mono<User> userMono = userClient.getUser(userId);

        return orderMono.flatMap(order -> {
            // Fetch only the product fields mobile needs (no full catalog data)
            List<String> productIds = order.getLines().stream()
                .map(OrderLine::getProductId)
                .toList();
            Mono<List<ProductSummary>> productsMono =
                productClient.getProductSummaries(productIds);  // Mobile-optimized endpoint

            return Mono.zip(userMono, productsMono)
                .map(tuple -> CheckoutSummary.builder()
                    .orderId(order.getId())
                    .total(order.getTotal())
                    .currency(order.getCurrency())
                    // Only fields mobile needs — not 50-field full product
                    .items(buildMobileItems(order.getLines(), tuple.getT2()))
                    .shippingAddress(tuple.getT1().getDefaultAddress())
                    .estimatedDelivery(order.getEstimatedDelivery())
                    .build());
        });
    }
}

// Response: lean payload for mobile
record CheckoutSummary(
    String orderId,
    BigDecimal total,
    String currency,
    List<MobileOrderItem> items,
    Address shippingAddress,
    LocalDate estimatedDelivery
) {}

record MobileOrderItem(
    String productId,
    String name,
    String thumbnailUrl,  // Small image only
    int quantity,
    BigDecimal price
) {}

Who Owns the BFF

WRONG: Platform team owns all BFFs
  → Bottleneck; frontend blocked on platform team; misaligned priorities

RIGHT: Frontend team owns their BFF
  Web team    → owns Web BFF
  iOS team    → owns Mobile BFF (or shared with Android)
  TV team     → owns TV BFF

  Internal services don't know about clients — BFF is the adapter layer

When NOT to Use BFF

• Only one client type (no differentiation needed)
• Team too small to maintain separate services
• Data needs are identical across clients
• GraphQL federation already solves the aggregation problem

API Gateway Products

Kong (Open Source)

# Declarative config (deck sync)
_format_version: "3.0"
_transform: true

services:
- name: order-service
  url: http://order-service:8081
  connect_timeout: 5000
  read_timeout: 30000
  write_timeout: 30000
  retries: 3
  routes:
  - name: orders
    paths: ["/api/v1/orders"]
    methods: ["GET", "POST", "PUT"]
  plugins:
  - name: jwt
    config:
      secret_is_base64: false
  - name: rate-limiting
    config:
      minute: 120
      policy: redis
  - name: request-transformer
    config:
      add:
        headers: ["X-Gateway-Version:2.0"]
      remove:
        headers: ["X-Internal-Debug"]
  - name: prometheus
    config:
      per_consumer: true

# Deploy Kong with Helm
helm repo add kong https://charts.konghq.com
helm install kong kong/kong \
  --set ingressController.enabled=true \
  --set postgresql.enabled=true \
  --set env.database=postgres

# Sync declarative config
deck sync --config kong.yaml

# Check routes
curl http://localhost:8001/routes | jq '.data[].paths'

AWS API Gateway

# serverless.yml: Lambda + API Gateway
functions:
  orders:
    handler: handlers/orders.handler
    events:
    - http:
        path: /orders
        method: POST
        authorizer:
          name: jwtAuthorizer
          type: TOKEN
          identitySource: method.request.header.Authorization
        throttling:
          maxRequestsPerSecond: 100
          maxConcurrentRequests: 50

  jwtAuthorizer:
    handler: handlers/authorizer.handler

Traefik (Kubernetes-Native)

# IngressRoute with middlewares
apiVersion: traefik.containo.us/v1alpha1
kind: IngressRoute
metadata:
  name: api-routes
  namespace: production
spec:
  entryPoints:
  - websecure
  routes:
  - match: Host(`api.example.com`) && PathPrefix(`/orders`)
    kind: Rule
    services:
    - name: order-service
      port: 8080
    middlewares:
    - name: auth-middleware
    - name: rate-limit-middleware
  tls:
    certResolver: letsencrypt   # Auto TLS from Let's Encrypt
---
apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:
  name: rate-limit-middleware
spec:
  rateLimit:
    average: 100      # 100 req/s
    burst: 200        # Allow burst up to 200
---
apiVersion: traefik.containo.us/v1alpha1
kind: Middleware
metadata:
  name: auth-middleware
spec:
  forwardAuth:
    address: http://auth-service:8080/validate
    authResponseHeaders:
    - X-User-Id
    - X-User-Role

Spring Cloud Gateway (Java / Reactive)

@Configuration
public class GatewayConfig {

    @Bean
    public RouteLocator routes(RouteLocatorBuilder builder,
                               JwtAuthFilter jwtFilter,
                               RateLimitFilter rateLimitFilter) {
        return builder.routes()
            // Order service routes
            .route("order-service", r -> r
                .path("/api/v1/orders/**")
                .filters(f -> f
                    .filter(jwtFilter)
                    .filter(rateLimitFilter)
                    .addRequestHeader("X-Gateway", "spring-cloud-gateway")
                    .addResponseHeader("X-Response-Time", "#{T(java.time.Instant).now()}")
                    .retry(config -> config
                        .setRetries(3)
                        .setStatuses(HttpStatus.BAD_GATEWAY, HttpStatus.SERVICE_UNAVAILABLE)
                        .setMethods(HttpMethod.GET)
                        .setBackoff(Duration.ofMillis(100), Duration.ofSeconds(2), 2, true))
                    .circuitBreaker(config -> config
                        .setName("order-service-cb")
                        .setFallbackUri("forward:/fallback/orders")))
                .uri("lb://order-service"))  // Load-balanced via service discovery

            // Product service with caching
            .route("product-service", r -> r
                .path("/api/v1/products/**")
                .and().method(HttpMethod.GET)
                .filters(f -> f
                    .filter(jwtFilter)
                    .requestRateLimiter(config -> config
                        .setRateLimiter(redisRateLimiter())
                        .setKeyResolver(userKeyResolver())))
                .uri("lb://product-service"))
            .build();
    }

    @Bean
    public RedisRateLimiter redisRateLimiter() {
        return new RedisRateLimiter(10, 20, 1);  // replenish rate, burst, requested tokens
    }
}

# application.yml fallback
spring:
  cloud:
    gateway:
      default-filters:
      - DedupeResponseHeader=Access-Control-Allow-Credentials Access-Control-Allow-Origin
      globalcors:
        corsConfigurations:
          '[/**]':
            allowedOrigins: "https://app.example.com"
            allowedMethods: ["GET", "POST", "PUT", "DELETE"]
            allowedHeaders: ["*"]
            allowCredentials: true

Authentication Patterns at the Gateway

PATTERN              HOW IT WORKS                           USE WHEN
-------              ------------                           --------
JWT validation       Verify signature + claims at gateway   Stateless; microservices
API key              Lookup key in Redis cache              Partner/B2B integrations
OAuth2 introspection Forward token to auth server          Opaque tokens; revocation needed
mTLS                 Client presents certificate            Service-to-service; zero-trust
Pass-through         Forward auth headers to service        Legacy services; avoid if possible

# OAuth2 token introspection with caching (avoid hitting auth server every request)
async def introspect_token(token: str, cache: Redis, auth_client: AuthClient) -> dict:
    cache_key = f"introspect:{hashlib.sha256(token.encode()).hexdigest()}"
    
    cached = await cache.get(cache_key)
    if cached:
        return json.loads(cached)
    
    result = await auth_client.introspect(token)
    
    if not result.get("active"):
        raise HTTPException(status_code=401, detail="Token inactive or expired")
    
    ttl = result.get("exp", 0) - int(time.time())
    if ttl > 0:
        await cache.setex(cache_key, min(ttl, 300), json.dumps(result))  # Cache max 5 min
    
    return result

Request Aggregation Patterns

REST Aggregation (Gateway fan-out)

# BFF: fetch from 3 services concurrently, merge response
import asyncio
import httpx

async def get_order_detail(order_id: str, user_id: str) -> dict:
    async with httpx.AsyncClient(timeout=5.0) as client:
        # Fan-out: all 3 calls in parallel
        order_task = client.get(f"http://order-service/orders/{order_id}")
        user_task = client.get(f"http://user-service/users/{user_id}")
        shipping_task = client.get(f"http://shipping-service/tracking/{order_id}")
        
        # Gather with timeout — partial response on failure
        results = await asyncio.gather(
            order_task, user_task, shipping_task,
            return_exceptions=True
        )
        
        order_resp, user_resp, shipping_resp = results
        
        response = {}
        
        if not isinstance(order_resp, Exception):
            response["order"] = order_resp.json()
        else:
            raise HTTPException(503, "Order service unavailable")  # Critical — fail
        
        if not isinstance(user_resp, Exception):
            response["user"] = user_resp.json()
        # else: non-critical — return without user data
        
        if not isinstance(shipping_resp, Exception):
            response["tracking"] = shipping_resp.json()
        else:
            response["tracking"] = {"status": "unavailable"}  # Graceful degradation
        
        return response

GraphQL Federation

# Each service owns its type — gateway federates automatically
# order-service schema:
type Order @key(fields: "id") {
  id: ID!
  total: Float!
  status: String!
  userId: ID!
  user: User          # Resolved by user-service federation
  products: [Product] # Resolved by product-service federation
}

# user-service schema:
type User @key(fields: "id") {
  id: ID!
  name: String!
  email: String!
}

# product-service schema:
type Product @key(fields: "id") {
  id: ID!
  name: String!
  price: Float!
}

# Client query — gateway handles federation automatically:
query GetOrderDetail($orderId: ID!) {
  order(id: $orderId) {
    id
    total
    status
    user {        # Fetched from user-service
      name
      email
    }
    products {    # Fetched from product-service
      name
      price
    }
  }
}

Observability at the Gateway

# Kong Prometheus plugin: RED metrics per route + consumer
plugins:
- name: prometheus
  config:
    per_consumer: true    # Metrics by API key / user
    status_code_metrics: true
    latency_metrics: true
    bandwidth_metrics: true

# Key metrics to alert on:
# kong_http_requests_total{route="orders",status="5xx"} > 1%  → alert
# kong_latency_ms{route="orders",quantile="0.99"} > 2000      → alert
# kong_upstream_target_health{target="order-svc"} == 0        → alert (all unhealthy)

# Inject distributed trace into forwarded request
import opentelemetry.propagate as propagate

def forward_with_tracing(request, upstream_url):
    with tracer.start_as_current_span("gateway.forward") as span:
        span.set_attribute("http.method", request.method)
        span.set_attribute("http.route", request.path)
        span.set_attribute("upstream.url", upstream_url)
        
        headers = dict(request.headers)
        # Inject W3C trace context into outgoing request
        propagate.inject(headers)
        
        response = httpx.request(
            request.method,
            upstream_url,
            headers=headers,
            content=request.body
        )
        span.set_attribute("http.status_code", response.status_code)
        return response

Anti-Patterns

Design Checklist

Gateway Configuration

• All routes require authentication (no anonymous internal routes)
• Rate limiting configured per consumer tier (free / standard / partner)
• Circuit breaker enabled for every upstream service
• Timeout set on all upstream routes (connect + read)
• SSL termination at gateway; internal traffic on private VPC
• Access logs enabled and shipped to observability platform
• Distributed trace context injected into all forwarded requests

BFF Design

• BFF owned by the frontend team that consumes it
• Response payloads contain only fields the client needs (no over-fetching)
• Fan-out calls to internal services are concurrent (not sequential)
• Partial response on non-critical service failure (graceful degradation)
• BFF has its own rate limit + circuit breaker (independent from shared gateway)

Rate Limiting

• Token bucket algorithm (allows burst, smooth average)
• Limits stored in Redis (distributed; works across gateway replicas)
• X-RateLimit-Limit, X-RateLimit-Remaining, X-RateLimit-Reset headers returned
• 429 Too Many Requests with Retry-After header on limit breach
• Different limits per API key tier

Authentication

• JWT signature verified (not just decoded) at gateway
• exp, iss, aud claims validated
• Verified claims forwarded as trusted headers to services
• Auth server introspection results cached in Redis (avoid hitting auth server per request)