Gke Security

GKE Security

This reference covers security configuration for GKE clusters. The golden path enforces a hardened security posture by default.

MCP Tools: get_cluster, check_k8s_auth, get_k8s_resource, apply_k8s_manifest, update_cluster

Golden Path Security Defaults

Workload Identity Federation

Workload Identity is the recommended way for pods to access Google Cloud APIs. It eliminates the need for static service account keys.

Setup

# 1. Create a Google Service Account (GSA)
gcloud iam service-accounts create <GSA_NAME> \
  --project <PROJECT_ID> \
  --display-name "Workload Identity SA" \
  --quiet

# 2. Grant IAM roles to the GSA
gcloud projects add-iam-policy-binding <PROJECT_ID> \
  --member "serviceAccount:<GSA_NAME>@<PROJECT_ID>.iam.gserviceaccount.com" \
  --role "<ROLE>" \
  --quiet

# 3. Create Kubernetes Service Account (KSA)
kubectl create namespace <NAMESPACE>
kubectl create serviceaccount <KSA_NAME> --namespace <NAMESPACE>

# 4. Bind KSA to GSA
gcloud iam service-accounts add-iam-policy-binding \
  <GSA_NAME>@<PROJECT_ID>.iam.gserviceaccount.com \
  --role roles/iam.workloadIdentityUser \
  --member "serviceAccount:<PROJECT_ID>.svc.id.goog[<NAMESPACE>/<KSA_NAME>]" \
  --quiet

# 5. Annotate KSA
kubectl annotate serviceaccount <KSA_NAME> \
  --namespace <NAMESPACE> \
  iam.gke.io/gcp-service-account=<GSA_NAME>@<PROJECT_ID>.iam.gserviceaccount.com

See assets/workload-identity-pod.yaml for a test pod.

Verification

kubectl run workload-identity-test \
  --image=gcr.io/google.com/cloudsdktool/cloud-sdk:slim \
  --serviceaccount=<KSA_NAME> --namespace=<NAMESPACE> \
  --rm -it -- gcloud auth list --quiet

Secret Manager Integration

The golden path enables Secret Manager with automatic rotation. Secrets are synced to Kubernetes Secrets.

# Verify Secret Manager is enabled on cluster
gcloud container clusters describe <CLUSTER_NAME> --region <REGION> \
  --format="value(secretManagerConfig.enabled)" \
  --quiet

# Enable if not already (Day-1 change)
gcloud container clusters update <CLUSTER_NAME> --region <REGION> \
  --enable-secret-manager \
  --secret-manager-rotation-interval=120s \
  --quiet

Mounting Secrets via CSI Volume (Deployment Example)

Once the Secret Manager add-on is enabled, workloads can mount secrets as volumes using the Secrets Store CSI driver. This requires two steps:

1. Define a SecretProviderClass to specify which secrets to retrieve from Secret Manager.
2. Mount the volume in a Deployment referencing that class.

[!IMPORTANT] Production Best Practice: Always demonstrate workload integrations (like Secret Manager CSI) using production-standard Deployment manifests rather than raw Pod manifests.

Step 1: Create the SecretProviderClass

apiVersion: secrets-store.csi.x-k8s.io/v1
kind: SecretProviderClass
metadata:
  name: app-secrets-provider
  namespace: default
spec:
  provider: gke  # Identifies GKE managed provider
  parameters:
    secrets: |
      - resourceName: "projects/<PROJECT_ID>/secrets/db-password/versions/latest"
        fileName: "db-password.txt"

Step 2: Mount the secret in a Deployment

apiVersion: apps/v1
kind: Deployment
metadata:
  name: secure-app
  namespace: default
spec:
  replicas: 2
  selector:
    matchLabels:
      app: secure-app
  template:
    metadata:
      labels:
        app: secure-app
    spec:
      serviceAccountName: secure-ksa  # Must be bound to GSA with Secret Manager Secret Accessor role
      containers:
      - name: app
        image: <IMAGE>
        volumeMounts:
        - name: secrets-volume
          mountPath: "/var/secrets"
          readOnly: true
      volumes:
      - name: secrets-volume
        csi:
          driver: secrets-store.csi.k8s.io
          readOnly: true
          volumeAttributes:
            secretProviderClass: "app-secrets-provider"

RBAC Hardening

The golden path disables insecure legacy RBAC bindings that grant broad access to system:authenticated and system:unauthenticated groups.

# Verify insecure bindings are disabled
gcloud container clusters describe <CLUSTER_NAME> --region <REGION> \
  --format="yaml(rbacBindingConfig)" \
  --quiet

Best practices for RBAC:

• Use namespace-scoped Roles over cluster-wide ClusterRoles
• Bind to specific Groups or ServiceAccounts, never to system:authenticated
• Audit permissions via MCP: check_k8s_auth(parent="...", verb="list", resourceType="pods", namespace="...") (or kubectl auth can-i --list --as=<user>)
• Review bindings via MCP: get_k8s_resource(parent="...", resourceType="clusterrolebinding") (or kubectl get clusterrolebindings,rolebindings --all-namespaces)

See the gke-multitenancy skill for enterprise RBAC planning and https://docs.cloud.google.com/kubernetes-engine/docs/best-practices/rbac.md.txt

Binary Authorization

Not enabled in golden path by default but recommended for production image provenance:

# Enable Binary Authorization
gcloud container clusters update <CLUSTER_NAME> --region <REGION> \
  --binauthz-evaluation-mode=PROJECT_SINGLETON_POLICY_ENFORCE \
  --quiet

Network Policies

Dataplane V2 (golden path) provides built-in Network Policy enforcement. Apply default-deny per namespace:

# MCP (preferred)
apply_k8s_manifest(parent="...", yamlManifest="<contents of default-deny-netpol.yaml>")

# kubectl fallback
kubectl apply -f ./assets/default-deny-netpol.yaml -n <NAMESPACE>

GKE Sandbox (gVisor)

For running untrusted workloads in an isolated sandbox:

# Enable on cluster (Standard clusters)
gcloud container clusters update <CLUSTER_NAME> --region <REGION> --enable-gke-sandbox --quiet

# Use in pod spec
# Add: runtimeClassName: gvisor

Pod Security Standards (Golden Path)

Pod Security Standards define three profiles that restrict what pods can do. The restricted profile is the golden path default for production namespaces.

Enforce via namespace labels (Pod Security Admission):

apiVersion: v1
kind: Namespace
metadata:
  name: production
  labels:
    pod-security.kubernetes.io/enforce: restricted
    pod-security.kubernetes.io/warn: restricted
    pod-security.kubernetes.io/audit: restricted

Gradual rollout strategy:

1. Start with warn + audit on existing namespaces to identify violations
2. Fix non-compliant workloads (remove privileged, hostNetwork, root user, etc.)
3. Enable enforce once all workloads pass

restricted blocks: running as root, privilege escalation, host networking/PID/IPC, host path volumes, and most capabilities. The golden path workload-identity-pod.yaml already complies.

Network Policy Logging (Recommended)

With Dataplane V2 (golden path), you can enable logging for Network Policy decisions. Not a golden path default -- recommended for security auditing.

gcloud container clusters update <CLUSTER_NAME> --region <REGION> \
  --enable-network-policy-logging \
  --quiet

This logs allowed and denied connections, useful for troubleshooting Network Policy rules and auditing traffic flows.

Common IAM Roles

The five most common predefined IAM roles for GKE:

Principle of least privilege: Start with roles/container.viewer or roles/container.developer and escalate only as needed. Avoid granting roles/container.admin broadly.

Service Accounts & Agents

• GKE Service Agent (service-<PROJECT_NUMBER>@container-engine-robot.iam.gserviceaccount.com): Automatically created. Manages nodes, networking, and cluster operations on your behalf. Do not remove or modify its permissions.
• Node Service Account: By default, nodes use the Compute Engine default service account. For production, create a dedicated SA with minimal permissions and assign it via node pool config.
• Workload Identity: The recommended way for pods to access Google Cloud APIs. Maps a Kubernetes ServiceAccount to a Google IAM ServiceAccount — see Workload Identity setup above.

Cross-Service Authentication Patterns

Common patterns for granting GKE workloads access to other Google Cloud services:

# Grant a GKE workload access to Cloud Storage
gcloud projects add-iam-policy-binding <PROJECT_ID> \
  --member "serviceAccount:<GSA_NAME>@<PROJECT_ID>.iam.gserviceaccount.com" \
  --role "roles/storage.objectViewer" \
  --quiet

# Grant a GKE workload access to Cloud SQL
gcloud projects add-iam-policy-binding <PROJECT_ID> \
  --member "serviceAccount:<GSA_NAME>@<PROJECT_ID>.iam.gserviceaccount.com" \
  --role "roles/cloudsql.client" \
  --quiet

# Grant a GKE workload access to Pub/Sub
gcloud projects add-iam-policy-binding <PROJECT_ID> \
  --member "serviceAccount:<GSA_NAME>@<PROJECT_ID>.iam.gserviceaccount.com" \
  --role "roles/pubsub.subscriber" \
  --quiet

In all cases, the GSA must be bound to a KSA via Workload Identity (see setup above). The pod then uses the KSA to authenticate as the GSA.