# Kubernetes Observability

> Capture and diagnose Zeabur Kubernetes service logs, events, and resource metrics directly with kubectl. Use whenever a request asks for service logs, pod output, CPU or memory usage, restart causes, or Kubernetes-side observability without the Zeabur backend.

- Skill: `zeabur/kubernetes-observability` (Agent Skill)
- Install (CLI): `npx skillmds@latest add zeabur/kubernetes-observability`
- Raw SKILL.md: https://api.skillmd.com/api/skills/zeabur/kubernetes-observability/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: DevOps & Infra
- Author: zeabur (https://skillmd.com/u/zeabur)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/zeabur/kubernetes-observability

---


Read [`../../references/kubernetes.md`](../../references/kubernetes.md) before collecting data. It defines the kubectl-only boundary, identity labels, and cluster capability discovery.

## Scope the observation

Require the environment and service IDs, time window, target container for multi-container Pods, and the question being answered. Resolve the workload and actual Pods by identity labels; do not select by a mutable display name.

```sh
kubectl -n "$NS" get pods -l "zeabur_type=user-service,zeabur_service_id=$ZEABUR_SERVICE_ID" -o wide
kubectl -n "$NS" get deploy,sts -l "zeabur_service_id=$ZEABUR_SERVICE_ID"
kubectl -n "$NS" get events --sort-by=.lastTimestamp
```

If the stricter label selector finds no Pod, retry with `zeabur_service_id=<service-id>` and identify whether an older workload is missing `zeabur_type`. Report that label drift; do not silently merge another service's data.

Complete this step only when every selected Pod and container is named, its phase and readiness are known, and the requested time window is bounded.

## Logs

Start with bounded, timestamped output and then examine the immediately previous container for crash loops:

```sh
kubectl -n "$NS" logs "$POD" -c "$CONTAINER" --since=1h --timestamps --tail=1000
kubectl -n "$NS" logs "$POD" -c "$CONTAINER" --previous --timestamps --tail=1000
kubectl -n "$NS" describe pod "$POD"
```

For a service-wide incident, collect the same bounded window from each selected Pod and container, preserve each Pod/container prefix, and correlate with Pod events. The local backend labels user workloads with `log-topic-key` (`logs.d.d-<deployment-id>` for Git deployments or `logs.m.s-<service-id>.e.e-<environment-id>` for marketplace services); surface it as a cluster label when present, but use Kubernetes logs as the independent capture path.

Redact environment values, tokens, connection strings, and Secret-backed paths from excerpts. Report timestamps, Pod/container identity, exit code, restart count, and the smallest relevant log window rather than an unbounded stream.

## Resource metrics

Check whether the Metrics API is available, then collect actual Pod usage and the configured resource contract together:

```sh
kubectl top pod -n "$NS" -l "zeabur_service_id=$ZEABUR_SERVICE_ID" --containers
kubectl -n "$NS" get pod "$POD" -o jsonpath='{range .spec.containers[*]}{.name}{" requests="}{.resources.requests}{" limits="}{.resources.limits}{"\n"}{end}'
kubectl -n "$NS" get pod "$POD" -o jsonpath='{range .status.containerStatuses[*]}{.name}{" restarts="}{.restartCount}{" ready="}{.ready}{"\n"}{end}'
```

`kubectl top` is an instantaneous Metrics Server view, not a historical time series. If it is unavailable, state that the cluster does not expose `metrics.k8s.io` to this context; still provide requested/limit values, restart counts, readiness, and events. Do not substitute a Zeabur metric endpoint.

For historical CPU or memory analysis, first discover a cluster-local Prometheus or VictoriaMetrics endpoint and the available labels. Use it only when its service, access method, and retention are explicitly known. The backend's Prometheus queries group cAdvisor CPU and memory series by `ZEABUR_ENVIRONMENT_ID` and `ZEABUR_SERVICE_ID`; validate those label names in the discovered target before issuing a query.

## Diagnose rollout and health failures

Use Kubernetes evidence in this order:

1. Deployment conditions and ReplicaSet revisions.
2. Pod scheduling and image-pull events.
3. Container `state`, `lastState`, exit code, and restart count.
4. Startup/readiness probe failures from `describe pod`.
5. Service EndpointSlices; an unready Pod must not be counted as serving traffic.
6. Bounded current and previous logs.

```sh
kubectl -n "$NS" rollout status deployment/$WORKLOAD --timeout=30s
kubectl -n "$NS" get deployment "$WORKLOAD" -o wide
kubectl -n "$NS" get rs -l "zeabur_service_id=$ZEABUR_SERVICE_ID"
kubectl -n "$NS" get endpointslice -l "kubernetes.io/service-name=$SERVICE_NAME" -o wide
```

## Completion

Return a time-bounded evidence table with Pod/container, phase/ready/restarts, CPU/memory use when available, requests/limits, and relevant events. Separate observed facts from inferences and name the next Kubernetes check when evidence is insufficient. Collection is complete when every selected Pod has been accounted for and no Secret value appears in the output.