aws-samples EKS Anywhere Diagnostics
When to use
Any EKS Anywhere investigation — cluster lifecycle, node provisioning, infrastructure provider issues, networking, curated packages, GitOps, cluster management service, IAM authentication, or backup/restore operations.
Investigation workflow
Step 1 — Collect and triage
eksctl anywhere version
kubectl get clusters -A --kubeconfig <mgmt-kubeconfig>
kubectl get machines -A --kubeconfig <mgmt-kubeconfig> -o wide
Step 2 — Domain deep dive
kubectl get clusters.anywhere.eks.amazonaws.com -A -o yaml --kubeconfig <mgmt-kubeconfig>
kubectl logs -n eksa-system deployment/eksa-controller-manager --tail=200 --kubeconfig <mgmt-kubeconfig>
kubectl get machinehealthchecks -A --kubeconfig <mgmt-kubeconfig>
Step 3 — Detailed investigation
kubectl get events -A --sort-by='.lastTimestamp' --kubeconfig <mgmt-kubeconfig> | tail -50
eksctl anywhere generate support-bundle --cluster <cluster-name> --kubeconfig <mgmt-kubeconfig>
kubectl get helmreleases -A --kubeconfig <workload-kubeconfig>
Read references/guardrails.md before concluding on any EKS Anywhere issue.
Tool quick reference
| Tool / API | When to use |
|---|---|
eksctl anywhere create cluster |
Create new EKS Anywhere cluster |
eksctl anywhere upgrade cluster |
Upgrade cluster version |
eksctl anywhere generate support-bundle |
Collect diagnostics bundle |
kubectl get clusters.anywhere |
List EKS Anywhere cluster resources |
kubectl get machines |
Check CAPI machine status |
kubectl get machinehealthchecks |
Review machine health checks |
kubectl get helmreleases |
Check curated package status |
Gotchas: EKS Anywhere
- EKS Anywhere uses Cluster API (CAPI) under the hood. Machine, MachineDeployment, and KubeadmControlPlane resources are CAPI objects, not EKS Anywhere-specific.
- The management cluster runs CAPI controllers that manage workload clusters. Losing the management cluster means losing the ability to manage workload clusters.
- vSphere provider requires specific permissions on vCenter — template VMs, resource pools, folders, and networks must be pre-configured.
- Bare metal provider uses Tinkerbell for provisioning. BMC/IPMI credentials and hardware inventory must be accurate.
- Cilium is the default CNI. Replacing it is NOT supported. Cilium policy enforcement mode affects network behavior.
- Curated packages require an AWS account and ECR token. Packages are pulled from ECR and require periodic token refresh.
- Flux GitOps integration manages cluster configuration declaratively. Manual changes to Flux-managed resources will be reverted.
Anti-hallucination rules
- Always cite specific cluster names, machine names, or controller logs as evidence.
- Management cluster and workload cluster are separate entities. Never conflate their kubeconfigs.
- CAPI resources (Machine, MachineDeployment) are different from EKS Anywhere resources (Cluster, VSphereDatacenterConfig). Never mix them.
- EKS Anywhere is NOT EKS. Do not suggest EKS-specific APIs (eks:DescribeCluster) for EKS Anywhere clusters.
- Spend no more than 2 minutes on any single hypothesis. Pivot if inconclusive.
14 runbooks
| Category | IDs | Covers |
|---|---|---|
| A — Cluster Lifecycle | A1-A2 | Creation failures, upgrade errors |
| B — Node Management | B1-B2 | Node provisioning, machine health checks |
| C — Providers | C1-C2 | vSphere provider, bare metal provider |
| D — Networking | D1-D2 | Cilium CNI, load balancer issues |
| E — Packages & GitOps | E1-E2 | Curated packages, Flux GitOps |
| F — Management & Auth | F1-F2 | Cluster management service, IAM authenticator |
| G — Backup & Restore | G1 | Backup and restore |
| Z — Catch-All | Z1 | General troubleshooting |