Agent Harness

Agent harness — what the agent is told, and how to audit what someone else told theirs

agent-orchestrator wires the loop. agent-evals proves it behaves. agent-interop gets it talking to other processes. This skill is the layer between them and the model: the prompt, the tools, and the choice of technique. The outside term for this ground is harness engineering — OpenAI's article of that name (openai.com/index/harness-engineering, read 2026-08-30) and Anthropic's harness-design guidance (anthropic.com/engineering/harness-design-long-running-apps, read 2026-08-30) both name this same layer, and its leverage is measured: on ARC-AGI-3, harness-level changes alone moved a fixed model from 13.3% to 38.3% while spending a sixth of the tokens (as reported 2026-08-30).

It runs in both directions. Building one and auditing one are the same checklist read forwards and backwards, which is why they live together here.

Rule zero — check the prompt first (a diagnostic heuristic, with exceptions)

The instinct when an agent misbehaves is to change the code. The sources this skill was built from pull the other way: "the biggest performance improvements often come from clearly explaining tool usage in the system prompt", and "even small refinements to tool descriptions can yield dramatic improvements." That is vendor guidance about where leverage OFTEN lives, not a measured share of defects — so it orders the DIAGNOSIS, never the verdict. The exceptions are the findings a prompt cannot touch: a deterministic race, a hardcoded secret, a timeout wired to the wrong operation — source-level invariant violations are code bugs, provable by reading, and no rewording treats them.

Before adding a retry, a router, or a sub-agent, check in this order:

1. Does the tool description say when to use it, not just what it does?
2. Does the system prompt name the vocabulary? An agent told to track status will invent pending and to-do and done and completed in the same run unless the allowed values are enumerated.
3. Does the agent know today's date? A model with a training cutoff will answer from memory rather than search unless the current date is injected.
4. Is the instruction flexible where it should be strict? "Use the tools in the order that makes most sense to you" is right while you are learning the task and wrong in production, where "you MUST execute a web search for each task" is what stops a step from being skipped.

Only then reach for architecture. Reaching for it first is how a prompt defect becomes a permanent structural cost.

Workflow or agent — decide this before anything else

An agent dynamically directs its own process. A workflow follows predefined code paths. The choice is not about sophistication; it is about whether the number of steps is knowable in advance.

Start at the simplest thing that works, and stop there. An agent adds latency, cost and a class of failure a workflow does not have — it needs trust in its own decisions. Pay for that only where a fixed path genuinely cannot be written.

The five workflow patterns, before you reach for autonomy

Orchestrator–workers versus routing is the distinction people get wrong. Routing picks from a known set. Orchestration invents the set per request. If you can enumerate the branches, you wanted routing and it is cheaper.

Static or dynamic — the second question, and it is not the same one

Having chosen a workflow, one thing is still open: is its shape known before it runs? A static graph has every node and edge decided up front; a dynamic one grows as nodes read their own output and decide what comes next.

Static first, always — go dynamic only after the static version hits a wall you can name, because dynamic is more powerful and much harder to control. But auditability is NOT the same axis as static structure — that conflates the plan drawn beforehand with the execution graph saved afterward. A run is auditable when its EXECUTION RECORD is complete: every node, edge and event that actually ran, the policy version in force, and deterministic bounds (budget / depth / node caps) with provenance. A static graph is the PREFERENCE because its executed shape usually matches the drawn one; a dynamic graph is auditable too when it keeps that record within those caps. What is never evidence is a design DIAGRAM on its own — "here is what I planned" is not "here is what happened", in either mode.

The six-row table, the rest of the model — the fake-edge test, the diamond, the checker node before a convergence — and what a host actually executes when it fans out are one home away: agent-orchestrator/references/graph-engineering.md. It is not restated here, because a decision table with two homes is one that will disagree with itself.

References

Each opens with its own Load this when line and a revision stamp — this material moves, and test/validate.py fails the build on a reference that does not say when it was read.

scripts/audit_agent.py — the mechanical half of the audit. It finds what is visible without understanding intent (an unbounded loop, a tool with no description, a swallowed tool error, a hardcoded model, a missing timeout) and prints the list of things it cannot see, so its silence is never read as a pass.

Auditing an agent system — the short version

The long version is references/audit.md. The shape:

1. Run the scanner first. It is cheap, and its blind-spot list tells you what the rest of the audit must cover by hand.
2. Walk the seven tracks — prompt, tools, control flow, context, failure, permission, evidence — and record a finding only with an observation attached.
3. Tier every recommendation by what backs it: measured here, documented upstream, or judgement.
4. Output a prioritized plan, not a score. A number tells nobody what to change on Monday. This is the same rule agent-evals applies to eval rubrics and seo-aeo-audit to sites.

The finding most audits surface first: the system has no evals. That is a finding about UNKNOWN RELIABILITY — every behavioural estimate downstream is unfalsifiable, including this audit's. It does NOT dissolve what is provable at the source: a demonstrable double charge, a hardcoded secret, a deterministic race keep their own findings and their own priority, set by the concrete harm — a general "no evals" never masks a specific proven harm.

Boundaries

Against agent-orchestrator. That skill owns the loop's plumbing: iteration guards, trimming, sub-agent dispatch, provider routing, memory layers, checkpoints. This one owns what the model is told. They meet at four seams, each crossing in exactly one place:

1. Describing a tool so the model picks the right one is this skill's references/tools.md; assembling the tool list per request from capability flags is the orchestrator's §3, which points here for the wording.
2. What the prompt says — altitude, vocabulary, enumerated statuses — is this skill's references/system-prompt.md; rebuilding that prompt per request, in the same pass as the tools, is the orchestrator's §10.
3. The shape of the work — the static/dynamic table, the fake-edge test, the checker before a convergence — has ONE home, and it is not here: agent-orchestrator/references/graph-engineering.md. This skill's static-or-dynamic section stops at the decision and links there for the model.
4. The context window: agent-orchestrator/references/context-engineering.md covers compaction — what to drop when the window fills — while this skill's system-prompt.md covers what to put there in the first place. Filling and emptying, two files.

Against agent-evals. That skill measures whether an agent behaves, from execution records. This one reviews how it was built, from its source and prompts. An audit that finds no evals hands over to it; an eval suite that keeps failing on the same axis hands back here.

Against agent-interop. MCP, A2A, the registry, gateways — the wire between processes. Tool descriptions are here; tool protocol is there.

Not covered: model choice and pricing (see the claude-api skill for Anthropic's), the wallet under resale (agent-orchestrator/references/llm-proxy-billing.md), and RAG retrieval quality, which is a search problem this skill only touches where it enters the prompt.

Checklist — a harness worth shipping

• Workflow-versus-agent decided deliberately, and the simpler option was actually tried
• Static-versus-dynamic decided too — static preferred for predictability; a run that must be auditable keeps a complete execution record (not merely a static shape)
• System prompt at the right altitude — heuristics, not hardcoded branches, not vague hope
• Every status, category and enum the agent must produce is enumerated in the prompt
• Today's date, and any other volatile context, injected rather than assumed
• Tools: a few high-impact ones, namespaced, each described as if to a new colleague
• Tool responses carry meaning, not identifiers, and are paginated or truncated by default
• Tool errors teach the next attempt instead of restating a stack trace
• One technique chosen per problem, with a reason — not ReAct because it was in a paper
• Sub-agents return distilled summaries, not transcripts
• The agent can be observed: which tool, which arguments, which observation, how many tokens
• An observable before the implementation, an eval before the prompt is tuned, or the tuning is folklore — only the corpus waits for production