Managing Seldon

Seldon Core Management Skill

Manage and monitor Seldon Core model deployments, inference graphs, and traffic management.

MANDATORY: Discovery-First Pattern

Always list existing deployments and their status before creating or modifying anything.

Phase 1: Discovery

#!/bin/bash

NAMESPACE="${SELDON_NAMESPACE:-default}"

echo "=== Seldon Core Version ==="
kubectl get deployment -n seldon-system seldon-controller-manager -o jsonpath='{.spec.template.spec.containers[0].image}' 2>/dev/null
echo ""

echo ""
echo "=== Seldon Deployments ==="
kubectl get seldondeployments -n "$NAMESPACE" -o json 2>/dev/null | jq -r '
    .items[] | "\(.metadata.name)\t\(.status.state // "Unknown")\t\(.status.replicas // 0) replicas\t\(.metadata.creationTimestamp[0:16])"
' | column -t

echo ""
echo "=== Deployment Pods ==="
kubectl get pods -n "$NAMESPACE" -l seldon-deployment-id --no-headers 2>/dev/null \
    | awk '{print $1"\t"$3"\t"$4"\t"$5}' | head -15

echo ""
echo "=== Istio Virtual Services (if present) ==="
kubectl get virtualservices -n "$NAMESPACE" -l seldon-deployment-id --no-headers 2>/dev/null | head -10

Core Helper Functions

#!/bin/bash

NAMESPACE="${SELDON_NAMESPACE:-default}"

# Seldon prediction helper
seldon_predict() {
    local deployment="$1"
    local data="$2"
    local endpoint="${SELDON_ENDPOINT:-http://localhost:8003}"
    curl -s -X POST "${endpoint}/seldon/${NAMESPACE}/${deployment}/api/v1.0/predictions" \
        -H "Content-Type: application/json" -d "$data"
}

# kubectl wrapper for Seldon CRDs
seldon_get() {
    local deployment="$1"
    kubectl get seldondeployment "$deployment" -n "$NAMESPACE" -o json 2>/dev/null
}

# Seldon deployment status check
seldon_status() {
    local deployment="$1"
    kubectl get seldondeployment "$deployment" -n "$NAMESPACE" -o jsonpath='{.status.state}' 2>/dev/null
}

Output Rules

• TOKEN EFFICIENCY: Target ≤50 lines per output
• Use kubectl with jq for structured CRD queries
• Never dump full SeldonDeployment specs -- extract key fields
• Test predictions with minimal sample payloads

Common Operations

Deployment Health Dashboard

#!/bin/bash
echo "=== Deployment Status Overview ==="
kubectl get seldondeployments -n "$NAMESPACE" -o json 2>/dev/null | jq -r '
    .items[] | "\(.metadata.name)\t\(.status.state // "Unknown")\t\(.status.deploymentStatus | to_entries | map("\(.key)=\(.value.replicas // 0)") | join(","))"
' | column -t

echo ""
echo "=== Unhealthy Deployments ==="
kubectl get seldondeployments -n "$NAMESPACE" -o json 2>/dev/null | jq -r '
    .items[] | select(.status.state != "Available") |
    "\(.metadata.name)\t\(.status.state // "Unknown")\t\(.status.description // "no details")"
' | column -t

echo ""
echo "=== Pod Health ==="
kubectl get pods -n "$NAMESPACE" -l seldon-deployment-id -o json 2>/dev/null | jq -r '
    .items[] | select(.status.phase != "Running") |
    "\(.metadata.name)\t\(.status.phase)\t\(.status.conditions[-1].message // "")"
' | column -t

Inference Graph Inspection

#!/bin/bash
DEPLOYMENT="${1:?Deployment name required}"

echo "=== Inference Graph: $DEPLOYMENT ==="
seldon_get "$DEPLOYMENT" | jq '{
    name: .metadata.name,
    state: .status.state,
    predictors: [.spec.predictors[] | {
        name: .name,
        traffic: .traffic,
        replicas: .replicas,
        graph: (.graph | {name: .name, type: .type, implementation: .implementation, children: [.children[]?.name]}),
        components: [.componentSpecs[]?.spec.containers[]? | {name: .name, image: .image}]
    }]
}'

A/B Testing Configuration

#!/bin/bash
DEPLOYMENT="${1:?Deployment name required}"

echo "=== Traffic Split: $DEPLOYMENT ==="
seldon_get "$DEPLOYMENT" | jq -r '
    .spec.predictors[] | "\(.name)\ttraffic=\(.traffic)%\treplicas=\(.replicas)\tmodel=\(.graph.name)"
' | column -t

echo ""
echo "=== A/B Test Metrics ==="
# Check if Seldon analytics is available
ANALYTICS_URL="${SELDON_ANALYTICS_URL:-http://localhost:8080}"
curl -s "${ANALYTICS_URL}/seldon-deploy/api/v1alpha1/analytics/prediction?deployment=${DEPLOYMENT}&namespace=${NAMESPACE}" 2>/dev/null \
    | jq '{
        total_predictions: .totalPredictions,
        by_predictor: .predictorPredictions,
        latency_p50: .latencyP50,
        latency_p99: .latencyP99
    }' 2>/dev/null || echo "Analytics endpoint not available -- check Seldon Deploy installation"

Canary Rollout Management

#!/bin/bash
DEPLOYMENT="${1:?Deployment name required}"
DRY_RUN="${2:-true}"

echo "=== Current Traffic Split ==="
seldon_get "$DEPLOYMENT" | jq -r '
    .spec.predictors[] | "\(.name)\ttraffic=\(.traffic)%\timage=\(.componentSpecs[0].spec.containers[0].image // "N/A")"
' | column -t

if [ "$DRY_RUN" = "true" ]; then
    echo ""
    echo "DRY RUN: To modify traffic, update the SeldonDeployment spec with new traffic percentages"
    echo "Ensure all predictor traffic values sum to 100"
else
    echo ""
    echo "Apply traffic changes via kubectl apply or Seldon Deploy API"
fi

Monitoring and Metrics

#!/bin/bash
DEPLOYMENT="${1:?Deployment name required}"

echo "=== Prediction Metrics (Prometheus) ==="
# Query Seldon metrics from Prometheus
PROM_URL="${PROMETHEUS_URL:-http://localhost:9090}"
curl -s "${PROM_URL}/api/v1/query?query=seldon_api_executor_server_requests_seconds_count{deployment_name=\"${DEPLOYMENT}\"}" 2>/dev/null \
    | jq -r '.data.result[]? | "\(.metric.method)\t\(.metric.code)\tcount=\(.value[1])"' | column -t

echo ""
echo "=== Latency (p99) ==="
curl -s "${PROM_URL}/api/v1/query?query=histogram_quantile(0.99,rate(seldon_api_executor_server_requests_seconds_bucket{deployment_name=\"${DEPLOYMENT}\"}[5m]))" 2>/dev/null \
    | jq -r '.data.result[]? | "\(.metric.method)\tp99=\(.value[1])s"' | column -t

echo ""
echo "=== Container Logs (last errors) ==="
kubectl logs -n "$NAMESPACE" -l seldon-deployment-id="$DEPLOYMENT" --tail=20 2>/dev/null \
    | grep -iE 'error|exception|fail' | tail -10

Safety Rules

• NEVER delete SeldonDeployments serving production traffic without draining connections first
• NEVER set traffic to 0% on all predictors -- at least one predictor must receive traffic
• Always validate traffic percentages sum to 100 before applying changes
• Test predictions with sample data after any deployment update
• Canary rollbacks: If canary shows errors, immediately shift traffic back to the stable predictor

Output Format

Present results as a structured report:

Managing Seldon Report
══════════════════════
Resources discovered: [count]

Resource       Status    Key Metric    Issues
──────────────────────────────────────────────
[name]         [ok/warn] [value]       [findings]

Summary: [total] resources | [ok] healthy | [warn] warnings | [crit] critical
Action Items: [list of prioritized findings]

Target ≤50 lines of output. Use tables for multi-resource comparisons.

Anti-Hallucination Rules

1. NEVER assume resource names — always discover via CLI/API in Phase 1 before referencing in Phase 2.
2. NEVER fabricate metric names or dimensions — verify against the service documentation or --help output.
3. NEVER mix CLI commands between service versions — confirm which version/API you are targeting.
4. ALWAYS use the discovery → verify → analyze chain — every resource referenced must have been discovered first.
5. ALWAYS handle empty results gracefully — an empty response is valid data, not an error to retry.

Counter-Rationalizations

Common Pitfalls

• Image pull errors: Private registries need imagePullSecrets in the SeldonDeployment spec -- missing secrets cause pods to hang in ImagePullBackOff
• Resource limits: Models without resource limits can OOM-kill other pods on the node -- always set memory limits
• Graph type confusion: ROUTER, COMBINER, and MODEL types have different behaviors -- incorrect type causes routing errors
• Istio sidecar injection: Seldon with Istio requires sidecar injection enabled -- missing sidecars break service mesh routing
• Prepackaged servers: Built-in servers (sklearn, xgboost, tensorflow) expect specific model artifact formats -- format mismatches cause load failures
• Init containers: Model download happens in init containers -- slow S3/GCS downloads cause long startup times
• Batch processing: Seldon batch jobs are separate from real-time serving -- do not mix configurations
• Version conflicts: Seldon operator version must be compatible with CRD versions -- check compatibility matrix before upgrading