Kubernetes Patterns

When to Use

• Use when: deploying a service to Kubernetes for the first time
• Use when: a pod is crashing (CrashLoopBackOff, OOMKilled, Pending)
• Use when: service is unreachable after deployment (probe misconfiguration)
• Use when: needing autoscaling based on CPU, memory, or custom metrics
• Do NOT use for: cluster setup — use managed k8s (EKS/GKE/AKS) or terraform-patterns
• Do NOT use for: service mesh (Istio/Linkerd) — separate concern

Production-Ready Deployment

apiVersion: apps/v1
kind: Deployment
metadata:
  name: api-service
  namespace: production
spec:
  replicas: 3
  selector:
    matchLabels:
      app: api-service
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 0      # zero-downtime: always have 3 pods serving
  template:
    metadata:
      labels:
        app: api-service
    spec:
      containers:
        - name: api
          image: myrepo/api-service:1.2.3   # always pin exact tag, never :latest
          ports:
            - containerPort: 3000

          # Resource budgets — REQUIRED or scheduler can't plan
          resources:
            requests:
              cpu:    "100m"      # guaranteed minimum
              memory: "128Mi"
            limits:
              cpu:    "500m"      # throttled at this, not killed
              memory: "256Mi"     # OOMKilled if exceeded

          # Probes — all three serve different purposes
          startupProbe:           # gives app time to boot (k8s ≥ 1.16)
            httpGet: { path: /health, port: 3000 }
            failureThreshold: 30
            periodSeconds: 5      # up to 150s to start

          readinessProbe:         # gates traffic routing
            httpGet: { path: /health/ready, port: 3000 }
            initialDelaySeconds: 0
            periodSeconds: 5
            failureThreshold: 3

          livenessProbe:          # restarts if stuck
            httpGet: { path: /health/live, port: 3000 }
            initialDelaySeconds: 30
            periodSeconds: 10
            failureThreshold: 3

          # Graceful shutdown
          lifecycle:
            preStop:
              exec:
                command: ["/bin/sh", "-c", "sleep 5"]   # drain in-flight requests

      terminationGracePeriodSeconds: 30

      # Security hardening
      securityContext:
        runAsNonRoot: true
        runAsUser: 1000
        readOnlyRootFilesystem: true

ConfigMap + Secret Injection

# ConfigMap — non-sensitive config
apiVersion: v1
kind: ConfigMap
metadata:
  name: api-config
data:
  LOG_LEVEL: "info"
  PORT: "3000"
  DB_HOST: "postgres.production.svc.cluster.local"

---
# Secret — sensitive values (base64-encoded)
apiVersion: v1
kind: Secret
metadata:
  name: api-secrets
type: Opaque
stringData:            # k8s base64-encodes automatically
  DB_PASSWORD: "$(vault kv get -field=password secret/prod/db)"
  API_KEY: "$(aws secretsmanager get-secret-value ...)"

---
# Inject into Pod
spec:
  containers:
    - name: api
      envFrom:
        - configMapRef: { name: api-config }    # all keys → env vars
        - secretRef:    { name: api-secrets }
      # Or individual key:
      env:
        - name: DB_PASS
          valueFrom:
            secretKeyRef: { name: api-secrets, key: DB_PASSWORD }

Horizontal Pod Autoscaler (HPA)

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
metadata:
  name: api-service-hpa
spec:
  scaleTargetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: api-service
  minReplicas: 2
  maxReplicas: 20
  metrics:
    - type: Resource
      resource:
        name: cpu
        target:
          type: Utilization
          averageUtilization: 60    # scale up when avg CPU > 60%
    - type: Resource
      resource:
        name: memory
        target:
          type: Utilization
          averageUtilization: 75

PodDisruptionBudget — Safe Node Drains

apiVersion: policy/v1
kind: PodDisruptionBudget
metadata:
  name: api-service-pdb
spec:
  minAvailable: 2       # at least 2 pods always running during node drain
  selector:
    matchLabels:
      app: api-service

Without PDB, kubectl drain kills all pods simultaneously → outage.

Debug Runbook

# Pod not starting
kubectl get pods -n production
kubectl describe pod <name> -n production   # look at Events section
kubectl logs <name> -n production --previous   # logs from crashed container

# Common causes:
# ImagePullBackOff  → wrong image tag or missing registry secret
# CrashLoopBackOff  → app crashes on startup; check logs
# OOMKilled         → increase memory limit
# Pending           → insufficient cluster resources; check node capacity

# Exec into running pod
kubectl exec -it <pod> -n production -- /bin/sh

# Port-forward for local debugging
kubectl port-forward svc/api-service 3000:3000 -n production

# Watch rolling update
kubectl rollout status deployment/api-service -n production
kubectl rollout undo deployment/api-service -n production   # rollback

Anti-Fake-Pass Rules

Before claiming a Kubernetes deployment is production-ready, you MUST show:

• resources.requests and resources.limits set on every container
• All three probes configured: startup, readiness, liveness
• Image pinned to exact digest or immutable tag — never :latest
• PodDisruptionBudget defined — prevents outage during node drain
• Secrets injected via secretRef — not hardcoded in deployment YAML
• runAsNonRoot: true in securityContext
• maxUnavailable: 0 in rolling update strategy for zero-downtime

Reference: gates/anti-fake-pass-gate.md