Kubernetes Platform

Kubernetes Platform

Treat the cluster like a product, not a toy. Workload teams should ship via a paved path (Helm/Kustomize + GitOps + policy) and never need to know which CNI you chose. Operators get day-2 covered: autoscaling, upgrades, security, multi-tenancy.

Stack Baseline (2026)

When to Use

• Multiple teams need isolated workloads on shared infra.
• Per-PR preview environments at scale.
• Need horizontal scaling beyond what PaaS offers.
• Standardizing on portable runtime across clouds.

Don't Use When

• One small service, one team — a managed PaaS (Cloud Run, Container Apps, Render) is cheaper end-to-end.
• No platform team to own day-2.

Prerequisites

• Cloud landing zone exists (../landing-zones/SKILL.md).
• IdP federation in place for cluster RBAC.
• Container registry with vulnerability scanning.
• CI/CD with OIDC to the cluster (no static kubeconfigs).

Inputs the Agent Should Gather

• Cloud(s), region(s), HA target.
• Tenant model: namespace per team vs cluster per environment vs vCluster.
• Compliance scope (PCI / HIPAA / FedRAMP).
• Workload mix: stateless web, batch, GPU, stateful.
• Existing tooling investments (Argo vs Flux, Helm vs Kustomize).

Instructions

1. Cluster topology

• ≥ 1 cluster per environment (prod, non-prod, sandbox); one cluster per region for prod HA.
• Three control-plane AZs; nodes in ≥ 2 AZs per nodegroup.
• Reserve a system nodegroup with taints for platform addons.
• Use Karpenter / Autopilot; avoid hand-tuned ASGs.

2. Multi-tenancy via namespaces + policy

# Kyverno: enforce required labels and resource limits per namespace
apiVersion: kyverno.io/v1
kind: ClusterPolicy
metadata: { name: tenant-baseline }
spec:
  validationFailureAction: Enforce
  rules:
    - name: require-owner-and-cost-center
      match: { any: [{ resources: { kinds: [Namespace] } }] }
      validate:
        message: "Namespace must have owner and cost-center labels"
        pattern:
          metadata:
            labels:
              owner: "?*"
              cost-center: "?*"
    - name: require-resource-limits
      match: { any: [{ resources: { kinds: [Pod] } }] }
      validate:
        message: "All containers need CPU and memory limits"
        pattern:
          spec:
            containers:
              - resources:
                  limits: { cpu: "?*", memory: "?*" }
                  requests: { cpu: "?*", memory: "?*" }

Combine with ResourceQuota, LimitRange, and NetworkPolicy default-deny per namespace.

3. GitOps as the only deploy path

# Argo CD ApplicationSet: one app per env per service via PR labels
apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata: { name: payments }
spec:
  generators:
    - matrix:
        generators:
          - list:
              elements:
                - { env: dev,  cluster: dev-eu  }
                - { env: prod, cluster: prod-eu }
          - git:
              repoURL: https://github.com/org/payments
              revision: HEAD
              files: [{ path: "deploy/{{env}}/values.yaml" }]
  template:
    metadata: { name: "payments-{{env}}" }
    spec:
      project: payments
      source:
        repoURL: https://github.com/org/payments
        path: charts/payments
        helm: { valueFiles: ["../deploy/{{env}}/values.yaml"] }
      destination: { server: "{{cluster}}", namespace: payments }
      syncPolicy: { automated: { prune: true, selfHeal: true } }

No kubectl apply from laptops in any environment.

4. Supply-chain trust

• Build images with reproducible builders (Docker BuildKit, ko, Buildpacks).
• Sign with cosign (keyless via OIDC).
• Generate SBOM (syft, cyclonedx).
• Admission verifies signature + provenance via Kyverno verifyImages or Sigstore policy-controller.
• Fail closed on unsigned images in prod.

5. Progressive delivery

# Argo Rollouts canary with analysis
apiVersion: argoproj.io/v1alpha1
kind: Rollout
metadata: { name: api }
spec:
  strategy:
    canary:
      steps:
        - setWeight: 10
        - pause: { duration: 5m }
        - analysis:
            templates: [{ templateName: success-rate }]
            args: [{ name: service, value: api }]
        - setWeight: 50
        - pause: { duration: 10m }

Analysis template queries Prometheus burn rate; auto-rollback on failure.

6. Day-2 operations

• Upgrades: N-1 always supported; rehearse on non-prod monthly.
• Backups: Velero for cluster state; provider snapshots for PVs.
• Drains: PodDisruptionBudgets on every workload.
• Observability: OTel Collector DaemonSet; one tracing/metrics/logging backend.
• Cost: OpenCost dashboard per namespace; Karpenter consolidation.

7. Security baseline

• Pod Security Admission restricted cluster-wide.
• Default-deny NetworkPolicy in every namespace.
• IRSA / Workload Identity / Azure Workload Identity — no node IAM.
• External Secrets Operator; never kubectl create secret from laptops.
• Falco / Tetragon for runtime detection routed to SIEM.

Common Pitfalls

Output Format

1. Cluster topology diagram (envs × regions × nodegroups).
2. Tenant model + namespace baseline (quotas, policies, network).
3. GitOps repo layout (app of apps / ApplicationSets).
4. Supply-chain pipeline: build → sign → SBOM → admit.
5. Progressive delivery template.
6. Day-2 runbook index (upgrade, backup, incident, drain).
7. Cost allocation model.

Abort Criteria

• No platform team capacity for day-2 — recommend managed PaaS.
• Single workload only — Kubernetes overhead exceeds value.
• No landing zone / IdP — fix prerequisites first.

Related Skills

• ../landing-zones/SKILL.md
• ../finops/SKILL.md — OpenCost, Karpenter consolidation
• ../../security/secure-development/SKILL.md — supply chain
• ../../engineer/devops-practices/ — GitOps, progressive delivery
• ../../architect/quality-attributes/slo-error-budgets/SKILL.md

Authoritative References

• Kubernetes docs — Pod Security Admission, NetworkPolicy, Gateway API
• CNCF — Argo, Flux, Cilium, Kyverno, OpenTelemetry projects
• AWS EKS Best Practices Guide
• Azure AKS Baseline architecture
• Google GKE Enterprise / Autopilot docs
• Sigstore + SLSA framework
• CIS Kubernetes Benchmark

Source: SwapnilPopat/ai-assistant-skills — distributed by TomeVault.