Microservices

Microservices Architecture

Purpose

Guide microservices decomposition, communication patterns, data ownership, and migration strategies.

Agent Protocol

Trigger

User request includes: microservice, micro-services, service decomposition, distributed system, saga, cqrs, event sourcing, service mesh.

Input Context

• Business domain description / bounded context map
• Current monolith architecture (if migrating)
• Team topology (Conway's Law)
• Non-functional requirements (latency, throughput, consistency, availability)
• Technology preferences (message broker, container platform, language)

Output Artifact

A markdown document containing:

• Service decomposition model (bounded contexts with responsibilities)
• Communication pattern selection (sync/async/event) per service pair
• Data ownership strategy (database per service, shared nothing)
• Infrastructure recommendations (service mesh, API gateway, message broker)
• Migration strategy (strangler fig, parallel run)

Response Format

Produce the artifact directly. No preamble, no postamble, no explanations. No filler, no hedging, no transitions. Strip articles a/an/the where unambiguous. Compress output — why use many token when few do trick. If monolith is appropriate, output Monolith recommended. Reason: [reason]. and stop.

Completion Criteria

• Decomposition bounded contexts explicitly mapped to business capabilities
• Each service pair has communication pattern documented with rationale
• Data consistency strategy for each transaction spanning services
• Infrastructure recommendations with concrete tools/versions
• Migration path with ordered phases

Max Response Length

4096 tokens

Decision Tree

Should You Use Microservices?

What is your situation?
  ├── Small team (<10 devs), early stage product, uncertain domain
  │   └── Monolith recommended. Reason: microservices add accidental complexity before you understand the domain.
  ├── Medium team, well-understood domain, scaling issues in monolith
  │   └── Decompose selectively: extract hot paths first (bounded contexts)
  ├── Large organization, multiple teams, clear domain boundaries
  │   └── Microservices aligned to team topology (Conway's Law)
  └── Migrating from monolith with growing complexity
      └── Strangler Fig: extract one bounded context at a time

How to Decompose?

What boundary defines this service?
  ├── Business capability (orders, payments, shipping)
  │   └── Standard approach: map to business functions
  ├── DDD subdomain (core, supporting, generic)
  │   └── Follow bounded contexts from domain modeling
  ├── Team topology (one team = one or more services)
  │   └── Conway's Law: services mirror communication structure
  └── Data ownership (candidate for extraction)
      └── If a data domain changes independently, it's a decomposition candidate

Communication Pattern Selection

What does the caller need?
  ├── Immediate response, strong consistency, low latency needed
  │   └── Synchronous (HTTP/gRPC) — but beware cascading failures
  ├── Fire-and-forget, eventual consistency OK, decouple in time
  │   └── Async (message queue / event) — resilient, scalable
  ├── Distributed transaction spanning 3+ services
  │   └── Saga orchestration — central coordinator
  ├── Guaranteed event publication DB → broker
  │   └── Transactional outbox — write event in same DB transaction
  └── Need to rebuild state from events or audit trail
      └── Event sourcing — store events as source of truth

Workflow

Step 1: Decompose by Bounded Context

Decomposition rules:

• Service must be independently deployable
• Service must own its data exclusively (no shared databases)
• Service must be team-sizable (2-pizza team)
• Service communication must be via network calls (no in-process)

Step 2: Select Communication Patterns

Saga variants:

• Choreography: Each service publishes events after local transaction. Lower complexity, harder to trace.
• Orchestration: Central coordinator tells each service what to do. Higher complexity, easier to trace, single point of failure.
• Selection rule: 3+ services in saga? Use orchestration. <3? Choreography is acceptable.

Step 3: Define Data Ownership

Data ownership rules:

• No service ever accesses another service's database directly
• No shared database between services (exception: read-only reference data)
• Data duplication is acceptable and expected (each service owns its view)

Step 4: Configure Service Discovery

Recommendation: Kubernetes-native (DNS for discovery, Service for load balancing). Only use external registry if running outside K8s.

Step 5: Implement Observability

Step 6: Choose Deployment Strategy

Step 7: Apply Resilience Patterns

Step 8: Implement Security

Step 9: Plan Migration from Monolith

1. Identify seams — areas of code that change independently
2. Extract read model first — read-only microservice serving cached/pre-computed data
3. Extract write model — feature flag to route writes to new service, dual-write during transition
4. Strangler — incrementally replace monolith endpoints with service endpoints
5. Monolith retirement — when all features migrated, decommission monolith

Step 10: API Versioning and Contracts

// Contract-first approach: define OpenAPI / protobuf before implementation
// Use consumer-driven contracts (CDC) to detect breaking changes

// API versioning strategies:
// 1. URL path: /v1/orders, /v2/orders
// 2. Header: Accept: application/vnd.myapp.v1+json
// 3. Query param: /orders?version=1
// Recommendation: URL path for major versions, additive field expansion for minor

Step 11: Cross-Cutting Concerns

Step 12: API Gateway Integration

// API Gateway (Kong / APISIX / Envoy) configuration pattern
// Route: /orders/* -> order-service:3001
// Route: /payments/* -> payment-service:3002
// Global: rate limit 1000 req/min per tenant, auth via JWT

// Gateway-level concerns:
// 1. Authentication — validate JWT before routing to service
// 2. Rate limiting — per-tenant, per-endpoint, burst allowance
// 3. Request validation — schema-based body validation at edge
// 4. Response transformation — strip internal headers, add CORS
// 5. Circuit breaking — gateway can fail fast before calling service

Step 13: Service Mesh Integration

# Istio VirtualService for traffic splitting (canary)
apiVersion: networking.istio.io/v1beta1
kind: VirtualService
metadata:
  name: order-service
spec:
  hosts:
  - order-service
  http:
  - match:
    - headers:
        x-canary: { exact: "true" }
    route:
    - destination:
        host: order-service
        subset: v2
      weight: 100
  - route:
    - destination:
        host: order-service
        subset: v1
      weight: 90
    - destination:
        host: order-service
        subset: v2
      weight: 10
# Service mesh provides: mTLS, traffic shifting, fault injection,
# circuit breaking, observability — without changing application code

Step 14: Database per Service Implementation

// Order Service — owns its PostgreSQL database
// Schema: orders, order_items, order_events
// No other service has direct DB access

// Payment Service — owns its PostgreSQL database
// Schema: payments, refunds, payment_methods

// Cross-service data access: via API calls only
// Example: Order Service needs payment status
// Solution: call `GET /payments/v1/orders/:orderId` — never query payment DB

Step 15: Contract Testing with Pact

// Consumer-driven contract test (Order Service -> Payment Service)
describe('Order Service - Payment API contract', () => {
  const provider = new Pact({
    consumer: 'OrderService',
    provider: 'PaymentService',
    port: 4000,
  });

  beforeAll(() => provider.setup());
  afterEach(() => provider.verify());
  afterAll(() => provider.finalize());

  it('should process payment and return transaction ID', async () => {
    await provider.addInteraction({
      state: 'payment can be processed',
      uponReceiving: 'a request to process payment',
      withRequest: {
        method: 'POST',
        path: '/v1/payments',
        headers: { 'Content-Type': 'application/json' },
        body: { orderId: '123', amount: 99.99, currency: 'USD' },
      },
      willRespondWith: {
        status: 200,
        headers: { 'Content-Type': 'application/json' },
        body: { transactionId: like('txn_abc123'), status: 'completed' },
      },
    });
    const result = await paymentClient.processPayment({ orderId: '123', amount: 99.99, currency: 'USD' });
    expect(result.transactionId).toBeDefined();
  });
});

API Composition Pattern

// API Gateway / BFF that aggregates multiple downstream services
// Instead of client making 4 requests, gateway does it server-side

interface ProductDetailsResponse {
  product: Product;
  inventory: Inventory;
  reviews: Review[];
  recommendations: Product[];
}

async function getProductDetails(productId: string): Promise<ProductDetailsResponse> {
  const [product, inventory, reviews, recommendations] = await Promise.all([
    fetchProductService(productId),
    fetchInventoryService(productId),
    fetchReviewService(productId),
    fetchRecommendationService(productId),
  ]);

  return { product, inventory, reviews, recommendations };
}

// Circuit breaker for resilient inter-service calls
class CircuitBreaker {
  private failures = 0;
  private lastFailureTime = 0;
  private state: 'CLOSED' | 'OPEN' | 'HALF_OPEN' = 'CLOSED';
  private readonly threshold = 5;
  private readonly timeout = 30000; // 30s

  async call<T>(fn: () => Promise<T>): Promise<T> {
    if (this.state === 'OPEN') {
      if (Date.now() - this.lastFailureTime > this.timeout) {
        this.state = 'HALF_OPEN';
      } else {
        throw new Error('Circuit breaker is OPEN');
      }
    }

    try {
      const result = await fn();
      this.failures = 0;
      this.state = 'CLOSED';
      return result;
    } catch (err) {
      this.failures++;
      this.lastFailureTime = Date.now();
      if (this.failures >= this.threshold) {
        this.state = 'OPEN';
      }
      throw err;
    }
  }
}

Service Discovery

# Kubernetes Headless Service for DNS-based discovery
# Services discover peers via SRV DNS lookups
apiVersion: v1
kind: Service
metadata:
  name: payment-service
spec:
  clusterIP: None  # Headless — returns pod IPs
  selector:
    app: payment-service
  ports:
    - name: grpc
      port: 50051
    - name: health
      port: 8080

// DNS-based service discovery with retry
import * as dns from 'dns/promises';

async function resolveService(name: string): Promise<string> {
  const records = await dns.resolveSrv(`_grpc._tcp.${name}.svc.cluster.local`);
  // Round-robin across healthy instances
  const target = records[Math.floor(Math.random() * records.length)];
  return `${target.name}:${target.port}`;
}

Production Considerations

Security

// Service-to-service authentication with mTLS
import { credentials, Metadata } from '@grpc/grpc-js';
import { readFileSync } from 'fs';

const rootCert = readFileSync('/etc/ssl/certs/ca-cert.pem');
const clientCert = readFileSync('/etc/ssl/certs/service-cert.pem');
const clientKey = readFileSync('/etc/ssl/certs/service-key.pem');

const channelCredentials = credentials.createSsl(rootCert, clientKey, clientCert);
const client = new PaymentServiceClient('payment-service:443', channelCredentials);

Anti-Patterns

Performance Benchmarks

Rules

• No shared databases between services — ever.
• Each service independently deployable with its own CI/CD.
• 3+ services in a saga? Use orchestration.
• Kubernetes-native discovery preferred over external registries.
• OpenTelemetry for all observability pillars.
• No service calls another service's database directly.
• Strangler Fig for monolith migration — no big-bang rewrites.
• Communication patterns documented per service pair with rationale.
• Always define API contracts before implementation (contract-first).
• Each service has at most one DB (shared-nothing).
• Prefer eventual consistency unless strong consistency is legally required.
• gRPC for high-throughput internal calls (< 2ms latency). HTTP for external or low-throughput internal.
• Rate limit at the gateway AND per-service for defense in depth.
• Use consumer-driven contracts to detect breaking API changes in CI.
• Every service must have independent backup and restore tested quarterly.

References

• references/communication-patterns.md — Communication Patterns
• references/data-patterns.md — Data Patterns
• references/decomposition-patterns.md — Decomposition Patterns
• references/microservices-communication.md — Microservices Communication
• references/microservices-observability.md — Microservices Observability
• references/microservices-testing.md — Microservices Testing

Handoff

Hand off to devops/containerization/SKILL.md for container orchestration setup. Hand off to backend/universal/event-driven/SKILL.md for detailed event-driven patterns. Hand off to backend/universal/database-patterns/SKILL.md for data consistency strategies.

Implementation Patterns

Observer Pattern for Event Handling

` interface EventObserver { onEvent(event: T): Promise; }

class EventBus { private observers: Set<EventObserver> = new Set(); subscribe(observer: EventObserver): void { this.observers.add(observer); } unsubscribe(observer: EventObserver): void { this.observers.delete(observer); } async emit(event: T): Promise { const results = Array.from(this.observers).map(o => o.onEvent(event)); await Promise.allSettled(results); } } `

Configuration-Driven Approach

config: defaults: timeout: 30s retryCount: 3 overrides: production: timeout: 60s retryCount: 5 development: timeout: 300s retryCount: 1

Production Considerations

Deployment Checklist

• Configuration validated against schema before startup
• Health check endpoints registered and monitored
• Graceful shutdown with draining period (30s timeout)
• Resource limits configured (CPU, memory, file descriptors)
• Log level set appropriate for environment
• Metrics endpoint secured and exposed
• Rate limiting configured per-tier
• TLS certificates valid and auto-renewing
• Database migrations run as separate deployment step
• Feature flags ready for gradual rollout

Monitoring and Alerting

Anti-Patterns

Performance Optimization

Caching Strategy

Cache hierarchy: L1 (in-memory local) → L2 (distributed Redis/Memcached) → L3 (CDN/Edge). Cache invalidation: TTL-based (simple, stale), event-based (complex, fresh), write-through (consistent, higher write latency), write-behind (fast writes, eventual consistency).

Resource Pooling

• Database connections: Pool of reusable connections (HikariCP, pgBouncer)
• HTTP connections: Keep-alive + connection pooling for external calls
• Thread pool: Bounded thread pools for async task execution

Profiling Methodology

1. Establish baseline with production traffic profile
2. Profile CPU with sampling profiler (pprof, perf, async-profiler)
3. Profile memory with heap dumps and allocation tracking
4. Profile I/O with strace/perf trace for syscall analysis
5. Profile latency with distributed tracing (OpenTelemetry)
6. Identify bottleneck, formulate hypothesis, implement fix
7. Re-profile to verify improvement, repeat

Security Considerations

Threat Modeling (STRIDE)

• Spoofing: Identity validation, authentication
• Tampering: Integrity checks, digital signatures
• Repudiation: Audit logs, non-repudiation
• Information disclosure: Encryption, access control
• Denial of service: Rate limiting, resource quotas
• Elevation of privilege: Principle of least privilege

Supply Chain Security

• Dependency scanning: Snyk, Dependabot, Trivy
• SBOM generation: CycloneDX or SPDX format
• Signed commits: GPG or SSH commit signing
• Artifact verification: Checksum validation, signature verification

Secrets Management

• Secrets never in code — always in secrets manager (Vault, AWS Secrets Manager)
• Rotation policy: Rotate database credentials every 90 days
• Access audit: Log every secrets access, alert on anomalies
• Encryption at rest and in transit for all secrets
• Principle of least privilege: each service gets only its own secrets

Rules

• Default-deny security posture — allow only explicitly required access.
• All inputs validated, all outputs encoded, all errors handled.
• Defend in depth — multiple layers of security controls.
• Fail securely — errors default to safe behavior.
• Log security-relevant events for audit and investigation.
• Keep dependencies updated — automate vulnerability scanning.
• Design for observability from day one, not as an afterthought.
• Document all architectural decisions with rationale.
• Review code for security, performance, and correctness before merging.