Konnect AI Gateway

Konnect AI Gateway triage

Goal

Diagnose the narrowest Konnect AI Gateway failure layer across request path, provider targeting, AI policy, and AI-runtime visibility.

Use this skill for AI Gateway operator workflows, not for generic prompt-engineering, provider SDK debugging, or model-quality evaluation.

Tool Selection

• Use the shared kong-konnect MCP server first for live inspection of control planes, Gateway entities, and LLM analytics surfaces.
• Prefer LLM-specific analytics when the user is asking about token usage, latency, or AI request health. Use the LLM analytics MCP/query surface when AI-runtime data is the real question rather than generic API analytics.
• Preserve the repository's chosen declarative toolchain for implementation: deck-gateway for Gateway-entity GitOps, terraform-konnect for HCL-managed Konnect AI Gateway resources, terraform-kong-gateway for self-managed HCL, and kongctl-declarative only when the surrounding Konnect repo already uses that path.
• If live Konnect state matters and kong-konnect MCP is not connected, say so early and continue with repo config or user-provided artifacts.
• Hand off early to konnect-gateway-triage when the blocker is generic gateway reachability, rollout, or route/service health rather than an AI-specific layer.

References To Load

Load the first matching reference. Do not load all three by default:

• references/provider-routing.md
  • Load when the main question is model/provider selection, route targeting, fallback, or provider-specific request behavior.
• references/guardrails-and-policy-intent.md
  • Load when traffic works technically but policy intent, prompt shaping, or governance behavior looks wrong.
• references/observability-and-cost-signals.md
  • Load when the question is about tokens, latency, AI-specific analytics, or operational visibility rather than request construction.

Inspection Order

1. Classify the report before choosing a layer

Place the first symptom in one branch:

• provider routing or model targeting
• service / route setup for AI traffic
• AI Proxy or AI Proxy Advanced behavior
• prompt or response guardrails
• prompt decoration or template behavior
• semantic cache behavior
• LLM usage, latency, or token visibility
• generic gateway reachability or config rollout that only happens to affect AI traffic

Do not flatten all AI Gateway issues into “the plugin is broken.”

If the complaint is fundamentally about ordinary route or service health, hand off to konnect-gateway-triage before diagnosing AI controls.

2. Prove the baseline AI traffic path

First prove:

• the AI request reaches the expected service and route
• the AI proxy layer is attached where the operator expects
• provider authentication expectations are clear
• the intended provider and model target are known, not assumed

Only then reason about higher-level AI policies such as prompt guards or templates.

If the request never reaches the provider path cleanly, stay below the guardrail layer and keep provider-routing or gateway-health branches in play.

3. Resolve provider or model targeting before policy

For governed LLM traffic, inspect the stack in this order:

• base AI proxying
• routing and model/provider selection
• authentication and access controls
• prompt decoration or template injection
• prompt / response guardrails
• semantic caching or other acceleration layers

This prevents chasing a guardrail symptom when the route or provider layer is wrong.

Load references/provider-routing.md when the operator is asking where traffic went, whether fallback occurred, or whether the wrong model/provider answered.

4. Evaluate policy layers only after proxying is proven

Once baseline proxying and targeting are credible, separate:

• prompt decoration or template injection errors
• prompt or response guardrail mismatches
• semantic cache behavior
• provider/model behavior outside Kong-side control

Load references/guardrails-and-policy-intent.md when the operator needs to separate technically valid traffic from policy-valid traffic.

5. Use LLM analytics for AI-specific runtime questions

If the question is about usage, tokens, latency, or cost-like behavior, prefer the LLM analytics surface over generic API analytics.

Use AI-specific observability to answer:

• whether requests are reaching the AI Gateway
• whether token usage aligns with expectations
• whether latency or error rates are concentrated by route, provider, or model

Load references/observability-and-cost-signals.md when the real problem is runtime visibility, token behavior, or latency concentration.

6. Distinguish Kong-side governance from provider behavior

When a request reaches the provider but still behaves incorrectly, separate:

• Kong-side routing and policy intent
• provider-side model behavior
• prompt construction effects
• authorization or key-management assumptions

Do not promise that every bad model response is a Gateway bug.

7. Return the narrowest AI failure domain

Classify the issue as:

• route or service setup problem
• AI proxy / provider configuration problem
• auth or secret-management problem
• prompt decoration / template problem
• guardrail policy problem
• semantic cache or acceleration problem
• analytics / visibility problem rather than traffic failure
• generic gateway health problem that should leave this skill

Konnect-Specific Gotchas

• AI Gateway is still a Gateway surface first: route, service, and plugin placement still matter.
• Prompt and response guardrails extend a working AI proxy path; they do not replace it.
• LLM analytics should be treated as a distinct runtime surface from generic API analytics.
• A technically successful provider response can still violate policy intent.
• Provider-side refusals, quotas, or model behavior can pass through a healthy AI Gateway path; prove the Kong-side intent before calling them Gateway defects.
• Quickstarts are useful for proving baseline capability, but they are not a substitute for a durable AI Gateway operating model.

Validation Checklist

Before answering, verify that you can state:

• which AI Gateway layer is failing
• whether baseline proxying works before higher-level AI controls
• which service/route and provider/model target the request is supposed to hit
• whether fallback or alternate provider selection is intended or accidental
• whether the issue is Kong-side routing, auth, guardrails, prompt shaping, caching, provider behavior, or analytics
• whether LLM analytics or another observability surface is the right next step
• which declarative tool skill should own the resulting change

Handoffs

• Use konnect-observability-triage when the main issue is dataset visibility rather than AI Gateway behavior itself.
• Use konnect-gateway-triage when the real blocker is generic Gateway connectivity or config rollout rather than an AI-specific layer.
• Use deck-gateway, terraform-konnect, terraform-kong-gateway, or kongctl-declarative when the operator wants to encode or apply the AI Gateway change as config.