K8s Troubleshooter
This skill provides expert-level Kubernetes diagnostic capabilities, leveraging advanced tools and established troubleshooting patterns.
Quick Start
- Verify Environment: Run
scripts/install_tools.shto ensurekubectl,stern, andk9sare available. - Find the Signal: Use
kubectl get pods -Aandkubectl get events -A --sort-by='.lastTimestamp'. - Analyze: Refer to the specialized reference guides below for in-depth diagnostics.
Workflow Decision Tree
1. Pod won't start?
Check status in kubectl get pods:
- ImagePullBackOff / ErrImagePull: See lifecycle.md.
- Pending: See lifecycle.md (check resources, taints, affinity).
- ContainerCreating: See lifecycle.md (check CNI, Volumes, Secrets).
2. Pod keeps crashing?
Check exit codes in kubectl describe pod:
- Exit Code 137 (OOMKill): Increase memory limits.
- Exit Code 1 / 255: Check application logs (use
sternfor easier tailing). - CrashLoopBackOff: See lifecycle.md.
3. Connectivity issues?
- DNS timeouts / slow resolution: See networking.md (check ndots and CoreDNS).
- Service unreachable: See networking.md (check selectors and endpoints).
- Ingress 502/503: See networking.md.
4. Storage problems?
- PVC Pending: See storage.md.
- Permission Denied: See storage.md.
5. Permission / Access issues?
- Forbidden error: See security.md (use
kubectl auth can-i). - Admission rejection: See security.md (check Pod Security Standards).
Specialized Diagnostics
- Networking: networking.md (DNS, Services, Ingress, Policies).
- Storage: storage.md (PVCs, Mounts, Permissions).
- Security: security.md (RBAC, Capabilities, Profiles).
- Node & Infrastructure: nodes.md (Kubelet, PLEG, CNI, Kube-Proxy).
- Pod Lifecycle: lifecycle.md (Startup, Runtime, Exit Codes).
- Quick Command Ref: troubleshooting.md.
Resources
- scripts/install_tools.sh: Automated installer for kubectl, stern, and k9s.
- references/: Detailed troubleshooting guides for different K8s domains.
- eval/EVAL.md: Instructions for running automated diagnostic evaluations of this skill.