Docker Compose Patterns

Docker Compose Advanced Patterns

Parent skill: docker-fundamentals. Basic Compose structure (services, volumes, networks, commands) is covered in docker-admin -- this skill covers advanced patterns only.

1. Profiles

Profiles define optional services that only start when explicitly activated.

services:
  app:
    image: myapp:1.2.0
    ports: ["8080:8080"]
  db:
    image: postgres:16-alpine
  pgadmin:
    image: dpage/pgadmin4:8.4
    profiles: [debug]
    ports: ["5050:80"]
  prometheus:
    image: prom/prometheus:v2.51.0
    profiles: [monitoring]
  node-exporter:
    image: prom/node-exporter:v1.7.0
    profiles: [debug, monitoring]       # starts with either profile

docker compose --profile debug up -d
docker compose --profile debug --profile monitoring up -d
docker compose up -d                    # starts only app + db (no profiles)

Services with no profiles key always start. A profiled service starts when at least one of its profiles is active.

2. Extends and Inheritance

# common.yaml -- shared base
services:
  base-web:
    init: true
    restart: unless-stopped
    deploy:
      resources:
        limits: { memory: 512M, cpus: "1.0" }
    logging:
      driver: json-file
      options: { max-size: "10m", max-file: "3" }

# compose.yaml
services:
  api:
    extends:
      file: common.yaml
      service: base-web
    image: myapi:2.1.0
    ports: ["8080:8080"]
  worker:
    extends:
      file: common.yaml
      service: base-web
    image: myworker:2.1.0
  worker-high-priority:
    extends:
      service: worker                   # same-file extends
    deploy:
      resources:
        limits: { memory: 2G, cpus: "2.0" }

Limitations: Cannot extend a service that uses depends_on, volumes_from, or links. Top-level networks/volumes are not inherited.

3. YAML Anchors and Merge Keys

Anchors (&), aliases (*), and merge keys (<<:) reduce duplication. Top-level x- keys are ignored by Compose and serve as anchor targets.

x-common-env: &common-env
  TZ: UTC
  LOG_LEVEL: info

x-healthcheck: &healthcheck
  interval: 30s
  timeout: 5s
  retries: 3

x-deploy: &deploy
  resources:
    limits: { memory: 512M, cpus: "1.0" }

services:
  api:
    image: myapi:3.0.0
    environment: { <<: *common-env, API_PORT: "8080" }
    healthcheck:
      <<: *healthcheck
      test: ["CMD", "curl", "-f", "http://localhost:8080/health"]
    deploy: *deploy
  worker:
    image: myworker:3.0.0
    environment: { <<: *common-env, WORKER_CONCURRENCY: "4" }
    healthcheck:
      <<: *healthcheck
      test: ["CMD", "curl", "-f", "http://localhost:9090/health"]
    deploy:
      <<: *deploy
      resources:
        limits: { memory: 1G, cpus: "2.0" }    # override nested key

Anchors vs extends: Anchors are pure YAML (work anywhere). extends is Compose-specific but supports cross-file inheritance.

4. Health-Dependent Startup

services:
  db:
    image: postgres:16-alpine
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U app -d appdb"]
      interval: 5s
      timeout: 3s
      retries: 10
      start_period: 10s
  redis:
    image: redis:7.2-alpine
    healthcheck:
      test: ["CMD", "redis-cli", "ping"]
      interval: 5s
      timeout: 3s
      retries: 5
  migrations:
    image: myapp:2.0.0
    command: ["python", "manage.py", "migrate"]
    depends_on:
      db: { condition: service_healthy }
  app:
    image: myapp:2.0.0
    depends_on:
      db: { condition: service_healthy }
      redis: { condition: service_healthy }
      migrations: { condition: service_completed_successfully }

Healthcheck Quick Reference

5. Multi-Environment Configuration

Use compose.override.yaml (auto-loaded) for dev overrides and named files for other environments.

# compose.yaml -- base                    # compose.override.yaml -- auto-loaded dev
services:                                  # services:
  app:                                     #   app:
    image: myapp:2.0.0                     #     build: { context: ., target: dev }
    environment:                           #     volumes: [".:/app", "/app/node_modules"]
      NODE_ENV: production                 #     ports: ["8080:8080", "9229:9229"]
  db:                                      #     environment: { NODE_ENV: development }
    image: postgres:16-alpine              #   db:
                                           #     ports: ["5432:5432"]

# compose.prod.yaml -- production overrides (explicit -f)
services:
  app:
    deploy: { replicas: 3, resources: { limits: { memory: 1G, cpus: "2.0" } } }
    restart: unless-stopped
    read_only: true
    tmpfs: [/tmp]

docker compose up -d                                        # dev (auto-loads override)
docker compose -f compose.yaml -f compose.prod.yaml up -d   # prod (skips override)
docker compose -f compose.yaml -f compose.prod.yaml config   # verify merged result

Environment Variable Substitution

services:
  app:
    image: ${REGISTRY:-docker.io}/myapp:${APP_VERSION:?APP_VERSION must be set}
    environment:
      DB_HOST: ${DB_HOST:-db}
      LOG_LEVEL: ${LOG_LEVEL:-info}

.env and env_file

# .env -- auto-loaded by Compose for interpolation in compose.yaml
COMPOSE_PROJECT_NAME=myproject
APP_VERSION=2.0.0

services:
  app:
    env_file:                           # loads vars INTO the container
      - .env.common
      - .env.${ENVIRONMENT:-dev}

Precedence (highest first): CLI -e > shell env > .env file > env_file > Dockerfile ENV.

6. Secrets and Configs

services:
  app:
    image: myapp:2.0.0
    secrets: [db_password, api_key]
    environment:
      DB_PASSWORD_FILE: /run/secrets/db_password
      API_KEY_FILE: /run/secrets/api_key
  db:
    image: postgres:16-alpine
    secrets: [db_password]
    environment:
      POSTGRES_PASSWORD_FILE: /run/secrets/db_password

secrets:
  db_password:
    file: ./secrets/db_password.txt
  api_key:
    file: ./secrets/api_key.txt

Secrets mount read-only at /run/secrets/<name>. Many official images (PostgreSQL, MySQL, MariaDB) natively support *_FILE env vars.

# Configs -- mount configuration files
services:
  nginx:
    image: nginx:1.27-alpine
    configs:
      - source: nginx_conf
        target: /etc/nginx/nginx.conf
        mode: 0444
configs:
  nginx_conf:
    file: ./nginx/nginx.conf

7. GPU Access

services:
  ml-training:
    image: pytorch/pytorch:2.2.0-cuda12.1-cudnn8-runtime
    deploy:
      resources:
        reservations:
          devices:
            - driver: nvidia
              count: all                # or count: 1, or device_ids: ["0","2"]
              capabilities: [gpu]

Requires NVIDIA driver + Container Toolkit + Docker nvidia runtime. Verify: docker run --rm --gpus all nvidia/cuda:12.1.0-base-ubuntu22.04 nvidia-smi.

8. Development Patterns

Watch Mode (Compose Watch)

services:
  web:
    build: { context: ., target: development }
    ports: ["3000:3000"]
    develop:
      watch:
        - action: sync                  # copy files, no rebuild
          path: ./src
          target: /app/src
          ignore: ["**/*.test.ts"]
        - action: sync+restart          # copy + restart container
          path: ./config
          target: /app/config
        - action: rebuild               # full image rebuild
          path: ./package.json

docker compose watch

Node.js Dev with Hot Reload

services:
  app:
    build: { context: ., target: development }
    volumes:
      - .:/app
      - /app/node_modules               # anonymous volume prevents host override
    ports: ["3000:3000", "9229:9229"]
    command: ["npx", "nodemon", "--inspect=0.0.0.0:9229", "src/index.ts"]

The anonymous volume /app/node_modules prevents the host bind mount from overwriting platform-specific native modules (e.g., esbuild, sharp).

Python Dev with Hot Reload

services:
  api:
    build: { context: ., target: development }
    volumes: [".:/app"]
    ports: ["8000:8000"]
    command: ["uvicorn", "app.main:app", "--reload", "--host", "0.0.0.0"]

9. Init Container Pattern

Use service_completed_successfully for tasks that must finish before the app starts.

services:
  db:
    image: postgres:16-alpine
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U app -d appdb"]
      interval: 5s
      timeout: 3s
      retries: 10
  db-migrate:
    image: myapp:2.0.0
    command: ["python", "manage.py", "migrate", "--noinput"]
    depends_on:
      db: { condition: service_healthy }
  db-seed:
    image: myapp:2.0.0
    command: ["python", "manage.py", "loaddata", "fixtures/reference.json"]
    depends_on:
      db-migrate: { condition: service_completed_successfully }
  app:
    image: myapp:2.0.0
    depends_on:
      db: { condition: service_healthy }
      db-seed: { condition: service_completed_successfully }
    ports: ["8080:8080"]

10. Sidecar Patterns

services:
  # --- Log collector sidecar (shared volume) ---
  app:
    image: myapp:2.0.0
    volumes: [app-logs:/var/log/app]
  log-collector:
    image: fluent/fluent-bit:3.0
    volumes: ["app-logs:/var/log/app:ro", "./fluent-bit.conf:/fluent-bit/etc/fluent-bit.conf:ro"]

  # --- Reverse proxy sidecar ---
  api:
    image: myapp:2.0.0
    expose: ["8080"]                    # no host port -- only through nginx
  nginx:
    image: nginx:1.27-alpine
    ports: ["443:443", "80:80"]
    volumes: ["./nginx.conf:/etc/nginx/nginx.conf:ro"]
    depends_on:
      api: { condition: service_healthy }

  # --- Debug sidecar (shared network namespace) ---
  debug:
    image: nicolaka/netshoot:v0.13
    profiles: [debug]
    network_mode: "service:app"         # shares app's network stack
    stdin_open: true
    tty: true

  # --- Build + serve (shared volume, init pattern) ---
  builder:
    image: node:22-alpine
    command: ["npm", "run", "build"]
    volumes: [".:/app", "assets:/app/dist"]
    working_dir: /app
  static-server:
    image: nginx:1.27-alpine
    volumes: ["assets:/usr/share/nginx/html:ro"]
    depends_on:
      builder: { condition: service_completed_successfully }

volumes:
  app-logs:
  assets:

11. Networking in Compose

services:
  api:
    image: myapi:2.0.0
    networks: [frontend, backend]       # accessible from both
  web:
    image: nginx:1.27-alpine
    networks: [frontend]                # cannot reach db
    ports: ["80:80"]
  db:
    image: postgres:16-alpine
    networks: [backend]

networks:
  frontend:
    driver: bridge
  backend:
    driver: bridge
    internal: true                      # no external/internet access

Aliases: networks: { backend: { aliases: [db, postgres] } } -- service reachable by multiple names.

External networks (cross-project): Project A creates networks: { shared: { name: shared-network } }. Project B joins with networks: { shared: { external: true, name: shared-network } }.

12. Resource Limits

services:
  app:
    image: myapp:2.0.0
    deploy:
      resources:
        limits: { memory: 512M, cpus: "1.0" }
        reservations: { memory: 256M, cpus: "0.5" }
      restart_policy: { condition: on-failure, delay: 5s, max_attempts: 3 }
    restart: unless-stopped
    logging:
      driver: json-file
      options: { max-size: "10m", max-file: "5" }
    pids_limit: 200

13. Production Patterns

services:
  app:
    image: myapp:2.0.0
    read_only: true
    tmpfs: ["/tmp:size=100M"]
    security_opt: ["no-new-privileges:true"]
    cap_drop: [ALL]
    cap_add: [NET_BIND_SERVICE]
    user: "1001:1001"
    init: true
    restart: unless-stopped
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:8080/health"]
      interval: 30s
      timeout: 5s
      retries: 3
      start_period: 20s
    deploy:
      resources:
        limits: { memory: 512M, cpus: "1.0" }
    logging:
      driver: json-file
      options: { max-size: "10m", max-file: "5" }

Rolling updates (Swarm): deploy: { replicas: 3, update_config: { parallelism: 1, delay: 10s, failure_action: rollback, order: start-first } }

14. Full Stack Example -- Node.js + PostgreSQL + Redis

Combines secrets, healthchecks, init containers, profiles, anchors, watch mode, and multi-environment overrides.

# compose.yaml -- production base
x-restart: &restart { restart: unless-stopped }

services:
  app:
    <<: *restart
    build: { context: ., target: production }
    ports: ["8080:8080"]
    environment: { NODE_ENV: production, DB_HOST: db, REDIS_URL: "redis://redis:6379" }
    secrets: [db_password]
    depends_on:
      db: { condition: service_healthy }
      redis: { condition: service_healthy }
      migrations: { condition: service_completed_successfully }
    init: true
    healthcheck:
      test: ["CMD", "curl", "-f", "http://localhost:8080/health"]
      interval: 15s
      timeout: 5s
      retries: 3
    deploy:
      resources:
        limits: { memory: 512M, cpus: "1.0" }
  db:
    <<: *restart
    image: postgres:16-alpine
    environment: { POSTGRES_DB: appdb, POSTGRES_USER: app, POSTGRES_PASSWORD_FILE: /run/secrets/db_password }
    secrets: [db_password]
    volumes: [db-data:/var/lib/postgresql/data]
    healthcheck:
      test: ["CMD-SHELL", "pg_isready -U app -d appdb"]
      interval: 5s
      timeout: 3s
      retries: 10
  redis:
    <<: *restart
    image: redis:7.2-alpine
    command: ["redis-server", "--maxmemory", "256mb", "--maxmemory-policy", "allkeys-lru"]
    healthcheck:
      test: ["CMD", "redis-cli", "ping"]
      interval: 5s
      timeout: 3s
      retries: 5
  migrations:
    build: { context: ., target: production }
    command: ["node", "dist/migrate.js"]
    secrets: [db_password]
    depends_on:
      db: { condition: service_healthy }
  pgadmin:
    image: dpage/pgadmin4:8.4
    profiles: [tools]
    ports: ["5050:80"]
    environment: { PGADMIN_DEFAULT_EMAIL: dev@local.dev, PGADMIN_DEFAULT_PASSWORD: admin }

volumes:
  db-data:
secrets:
  db_password:
    file: ./secrets/db_password.txt

# compose.override.yaml -- dev overrides (auto-loaded)
services:
  app:
    build: { context: ., target: development }
    volumes: [".:/app", "/app/node_modules"]
    ports: ["3000:3000", "9229:9229"]
    environment: { NODE_ENV: development, DATABASE_URL: "postgresql://dev:devpass@db:5432/devdb" }
    restart: "no"
    develop:
      watch:
        - action: sync
          path: ./src
          target: /app/src
        - action: rebuild
          path: ./package.json
  db:
    ports: ["5432:5432"]
  redis:
    ports: ["6379:6379"]

docker compose up -d                       # dev (auto-loads override)
docker compose --profile tools up -d       # dev + pgadmin
docker compose watch                       # hot reload
docker compose -f compose.yaml up -d       # prod (skips override)
docker compose down -v && docker compose up -d  # reset DB

Anti-Patterns

Related Skills