Managing Istio Deep

Istio Deep Skill

Advanced Istio management including Envoy debugging, multi-cluster mesh, Wasm extensions, and advanced traffic policies.

Core Helper Functions

#!/bin/bash

istio_cmd() {
    istioctl "$@" 2>/dev/null
}

istio_get() {
    local resource="$1" ns="${2:---all-namespaces}"
    if [ "$ns" = "--all-namespaces" ]; then
        kubectl get "$resource" -A -o json 2>/dev/null
    else
        kubectl get "$resource" -n "$ns" -o json 2>/dev/null
    fi
}

# Envoy admin API via port-forward
envoy_admin() {
    local pod="$1" ns="${2:-default}" path="${3:-stats}"
    kubectl exec "$pod" -n "$ns" -c istio-proxy -- curl -s "localhost:15000/$path" 2>/dev/null
}

MANDATORY: Discovery-First Pattern

Phase 1: Discovery

#!/bin/bash

echo "=== Istio Version & Revision ==="
istioctl version --short 2>/dev/null
kubectl get mutatingwebhookconfigurations -l app=sidecar-injector -o json 2>/dev/null | jq -r '
    .items[] | "\(.metadata.name)\tRevision: \(.metadata.labels["istio.io/rev"] // "default")"
'

echo ""
echo "=== Control Plane Health ==="
kubectl get pods -n istio-system -o json 2>/dev/null | jq -r '
    .items[] | "\(.metadata.name)\t\(.status.phase)\tRestarts: \(.status.containerStatuses[0].restartCount // 0)\tCPU: \(.spec.containers[0].resources.requests.cpu // "n/a")"
' | column -t

echo ""
echo "=== Mesh Configuration ==="
kubectl get configmap istio -n istio-system -o json 2>/dev/null | jq -r '.data.mesh' | head -20

echo ""
echo "=== Proxy Sync Status ==="
istioctl proxy-status 2>/dev/null | head -20

echo ""
echo "=== EnvoyFilter Resources ==="
kubectl get envoyfilters -A -o json 2>/dev/null | jq -r '
    .items[] | "\(.metadata.name)\t\(.metadata.namespace)\t\(.spec.configPatches | length) patches"
' | column -t | head -10

echo ""
echo "=== WasmPlugin Resources ==="
kubectl get wasmplugins -A -o json 2>/dev/null | jq -r '
    .items[]? | "\(.metadata.name)\t\(.metadata.namespace)\t\(.spec.url[:50])"
' | column -t | head -10

Phase 2: Analysis

#!/bin/bash
POD="${1:?Pod name required}"
NS="${2:-default}"

echo "=== Envoy Config Dump Summary ==="
envoy_admin "$POD" "$NS" "config_dump" | jq '{
    listeners: (.configs[] | select(.["@type"] | test("ListenersConfigDump")) | .dynamic_listeners | length),
    clusters: (.configs[] | select(.["@type"] | test("ClustersConfigDump")) | .dynamic_active_clusters | length),
    routes: (.configs[] | select(.["@type"] | test("RoutesConfigDump")) | .dynamic_route_configs | length)
}'

echo ""
echo "=== Envoy Cluster Health ==="
envoy_admin "$POD" "$NS" "clusters?format=json" | jq '
    .cluster_statuses[:10] | .[] | {
        name: .name, health: .host_statuses[0].health_status.eds_health_status,
        outlier_detected: .host_statuses[0].health_status.failed_outlier_check
    }'

echo ""
echo "=== Circuit Breaker Status ==="
istio_get destinationrules "$NS" | jq '
    .items[] | select(.spec.trafficPolicy.connectionPool or .spec.trafficPolicy.outlierDetection) | {
        name: .metadata.name, host: .spec.host,
        connectionPool: .spec.trafficPolicy.connectionPool,
        outlierDetection: .spec.trafficPolicy.outlierDetection
    }' | head -20

echo ""
echo "=== Rate Limiting Config ==="
istio_get envoyfilters "$NS" | jq '
    .items[] | select(.spec.configPatches[].applyTo == "HTTP_FILTER") | {
        name: .metadata.name,
        patches: [.spec.configPatches[] | .applyTo]
    }' | head -15

echo ""
echo "=== Proxy Resource Usage ==="
kubectl top pod "$POD" -n "$NS" --containers 2>/dev/null | grep istio-proxy

Output Rules

• TOKEN EFFICIENCY: Target <=50 lines per output
• Never dump full Envoy config -- extract relevant sections only
• Use istioctl proxy-config for targeted proxy inspection

Safety Rules

• Read-only by default: Use analyze, proxy-status, proxy-config for inspection
• EnvoyFilter changes can break all mesh traffic if misconfigured
• WasmPlugin errors cause sidecar crashes -- test in staging first
• Multi-cluster changes affect cross-cluster service discovery

Output Format

Present results as a structured report:

Managing Istio Deep Report
══════════════════════════
Resources discovered: [count]

Resource       Status    Key Metric    Issues
──────────────────────────────────────────────
[name]         [ok/warn] [value]       [findings]

Summary: [total] resources | [ok] healthy | [warn] warnings | [crit] critical
Action Items: [list of prioritized findings]

Target ≤50 lines of output. Use tables for multi-resource comparisons.

Anti-Hallucination Rules

1. NEVER assume resource names — always discover via CLI/API in Phase 1 before referencing in Phase 2.
2. NEVER fabricate metric names or dimensions — verify against the service documentation or --help output.
3. NEVER mix CLI commands between service versions — confirm which version/API you are targeting.
4. ALWAYS use the discovery → verify → analyze chain — every resource referenced must have been discovered first.
5. ALWAYS handle empty results gracefully — an empty response is valid data, not an error to retry.

Counter-Rationalizations

Common Pitfalls

• Envoy config size: Full config dumps can be massive; always filter by section
• EnvoyFilter ordering: Multiple EnvoyFilters can conflict; priority field controls order
• Revision-based upgrades: Canary control plane upgrades require careful namespace relabeling
• Ambient mesh: Ztunnel-based datapath has different debugging than sidecar-based
• Memory limits: Envoy sidecars with large configs can OOM; check proxy resource limits