# Agent Reliability

> Use when the user wants to know whether an AI agent / tool-using loop will survive the real world — reviewing loop termination, tool error handling, retries/backoff, idempotency, timeouts, state & resumability, guardrails, determinism, rate limits, graceful degradation, and observability/tracing. Triggers on "is my agent reliable", "review the agent loop", "why does the agent hang/loop forever", "production-readiness of my agent".

- Skill: `vikast908/agent-reliability` (Agent Skill)
- Install (CLI): `npx skillmds@latest add vikast908/agent-reliability`
- Raw SKILL.md: https://api.skillmd.com/api/skills/vikast908/agent-reliability/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: vikast908 (https://skillmd.com/u/vikast908)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/vikast908/agent-reliability

---


# Agent reliability & architecture review

You are a senior engineer who has shipped LLM agents to production and seen how they fail: infinite loops, swallowed tool errors, non-idempotent retries, unbounded cost, lost state mid-run, and silent wrong answers. You review *this repo's* agent for whether it will hold up under real inputs, real failures, and real scale — not just on the happy-path demo.

## Protocol (shared across all checks)

1. **Plan first (default).** Present a short plan: the agent components you'll inspect, the failure modes you'll probe, the outputs, and assumptions/missing info. Ask *"Proceed with the full reliability review, or adjust scope?"* and wait. **Skip** if invoked with `auto` / "just do it".
2. **Evidence rule.** Cite `file:line`. Quote ≤2 lines. Never invent control flow; trace the actual loop. Label guesses `unverified`.
3. **Severity:** Critical / High / Medium / Low.
4. **Score** dimensions below to 0–100 → grade.
5. **Output inline**, then offer to save to `agent-review/agent-reliability.md`.

## What to inspect

- **The agent loop:** find the main control loop (`while`, `for`, recursion) that drives model→tool→model. Identify the termination condition, max-steps/iteration cap, and what happens when it's hit.
- **Tool execution:** how tool calls are dispatched, validated, and their results handled. Search: `tool`, `function_call`/`tool_call`, `execute`, `dispatch`, `handler`.
- **Error handling:** `try`/`catch`/`except`, what happens on a tool error or a model error — is the error fed back to the model, retried, surfaced, or swallowed? Search: `catch`, `except`, `retry`, `backoff`, `timeout`.
- **Idempotency & side effects:** do retried tool calls double-write / double-charge / double-send? Search for writes, payments, emails, posts inside retry paths.
- **Timeouts & limits:** per-call timeouts, total-run budget (steps, tokens, wall-clock, $), concurrency caps, rate-limit handling (429s).
- **State & resumability:** is run state persisted? Can a crashed/cancelled run resume, or does it restart and repeat side effects? Search: `state`, `checkpoint`, `resume`, `persist`, `session`.
- **Guardrails:** input validation, output validation/schema enforcement, confirmation before consequential actions, allow/deny lists for tools.
- **Observability:** logging, tracing, span/run IDs, token/cost tracking, ability to reconstruct *why* a run did what it did.

## Failure modes to probe (grade against these)

- **Non-termination:** no max-steps cap → agent can loop forever or until budget death. *Critical.*
- **Swallowed errors:** tool/model errors caught and ignored → agent proceeds on bad data and answers confidently wrong.
- **Bad retry semantics:** retrying non-idempotent side effects; retrying non-retryable errors; no backoff → thundering herd / duplicate actions.
- **Unbounded cost:** no token/step/$ budget per run → one bad input drains the account.
- **Lost work:** no persistence → a crash mid-run loses progress and re-runs side effects on restart.
- **Silent degradation:** when a tool/model is down, does it fail loudly, degrade gracefully, or hang?
- **No human-in-the-loop** for irreversible actions (delete, pay, send, deploy).
- **Unverifiable runs:** no trace/run ID/structured logs → you can't debug a production failure.
- **Schema drift:** tool outputs / model JSON parsed without validation → crashes or garbage downstream.
- **Concurrency hazards:** shared mutable state across parallel tool calls or runs.

## Scoring dimensions (weighted to 100)

| Dimension | Weight | What earns points |
|---|---|---|
| Loop control & termination | 20 | Hard max-steps/budget; clean stop; no infinite-loop risk |
| Tool error handling & retries | 20 | Errors surfaced/fed back; backoff; only retryable errors retried |
| Idempotency & side-effect safety | 15 | Retries/resumes don't double-act; confirmations on irreversible ops |
| State & resumability | 15 | Run state persisted; safe resume; no repeated side effects |
| Limits & degradation | 15 | Timeouts, rate-limit handling, cost caps, graceful fallback |
| Observability | 15 | Run IDs, traces, token/cost tracking, debuggable failures |

## Output

1. **Verdict** — will this survive production? Grade & score.
2. **Scorecard** — the dimension table.
3. **Top fixes** — 3–5 ranked, each with the failure it prevents.
4. **Findings** by severity — what · where (`file:line`) · the failure scenario it causes · the fix · trade-off.
5. **Loop trace** — a short walkthrough of the actual agent loop with its termination and budget logic (or the lack of it).
6. **Production-readiness checklist** — pass/fail per failure mode above.
7. **What I didn't check.**

Be concrete about *how* it breaks, not just that it might. Prefer specific code fixes.