Audit AI Agents

Audit AI Agents

Overview

Treat an agent audit as an evidence-producing investigation of a socio-technical system, not as a single benchmark score. Map the audited boundary, threat model, claims, controls, traces, tests, and residual uncertainty across the model, harness, tools, memory, identities, environment, humans, and operational process.

This skill combines literature research, code inspection, audit design, evidence grading, incident-aware analysis, and research-gap generation. It favors primary artifacts and reproducible tests, records negative and conflicting evidence, and states what the available access cannot establish.

Route the request

1. For a literature review or frontier-company map, read references/taxonomy.md, references/frontier-labs.md, references/literature.md, and references/update-protocol.md.
2. For an audit plan or execution, read references/taxonomy.md and references/audit-playbook.md; add references/codebases.md when selecting tools or benchmarks.
3. For production monitoring, evidence packages, or an incident, read references/audit-playbook.md and the provenance/forensics sections of references/literature.md.
4. For a paper, benchmark, or repository review, read references/codebases.md and the evidence-extraction protocol in references/update-protocol.md.
5. For new research ideas, first complete the evidence map, then read references/research-gaps.md. Never generate novelty claims from memory alone.
6. Query the curated corpus with python3 scripts/query_corpus.py --help. Treat it as a dated seed set, not a substitute for a current search.

Establish the audit contract

Before collecting evidence, write a compact contract containing:

• Decision: the concrete release, deployment, procurement, incident, research, or assurance decision the audit must support.
• System boundary: exact model/version, system prompt, harness, tools and protocols, identities and permissions, memory/RAG, environment, external services, human approvals, monitors, and logging stack.
• Stage: design review, pre-deployment, runtime/continuous, post-incident, or third-party assurance.
• Access: black-box; gray-box with prompts, traces, logs, training data, or internals; or white-box with weights/activations. List unavailable evidence.
• Threat model: actor, objective, capability, knowledge, access, persistence, collusion, and acceptable impact threshold.
• Claims: separate capability, propensity, control effectiveness, security, compliance, and incident claims.
• Freshness date: search cutoff, model build, harness commit, policy version, tool versions, and audit environment.

If the boundary or decision is missing, ask only for information that would materially change the test plan. Otherwise state a narrow assumption and proceed.

Run the evidence workflow

1. Build an audit register

Create rows for each relevant combination of system layer, risk, lifecycle stage, expected control, evidence source, test, metric, owner, status, and residual uncertainty. Use the dimensions and risk families in references/taxonomy.md.

Do not collapse these activities:

• capability evaluation: what the agent can do;
• propensity or alignment audit: what it is inclined to do;
• AI-control evaluation: whether harm remains containable even if the agent is untrusted;
• security red team: whether an adversary can manipulate or compromise the system;
• runtime monitoring: what the deployed system did or is doing;
• assurance audit: whether control design and operation meet stated obligations;
• incident forensics: what happened, why, and with what evidence integrity.

2. Search systematically

Follow references/update-protocol.md. Start with official research pages, papers, system cards, standards, incident reports, repositories, datasets, and evaluation code. Then use surveys and citation graphs for coverage. Record exact queries, repositories, dates, inclusion decisions, and unresolved names.

Never promise literally “all articles.” Report a search-complete set relative to named databases, organizations, query families, languages, artifact types, and cutoff date.

3. Extract comparable evidence

For every included artifact capture:

• exact claim and audit object;
• authors/organization, date, artifact status, and funding or first-party relationship;
• model, harness, environment, access level, threat model, and adversary;
• task construction, sample size, seeds/runs, baselines, controls, scorer, and human review;
• capability ceiling, attack success, monitor discrimination, safety and utility, severity, uncertainty, and audit cost where reported;
• realism, evaluation-awareness, contamination, independence, transfer, and reproduction limits;
• code/data availability, license, infrastructure, credentials, and last maintenance signal.

Label findings confirmed, supported, indicative, company-reported, disputed, or unknown. A company blog can be primary evidence of what that company reports, but not independent validation of the underlying safety claim.

4. Inspect implementation, not just prose

When code is available, inspect entry points, dependency locks, environment isolation, privilege boundaries, tool mocks, scorer implementation, hidden-test protection, transcript format, resume behavior, randomization, seed handling, telemetry, and expected compute/API cost. Run a safe smoke test when feasible. Record deviations between the paper, current branch, released data, and claimed setup.

Prefer synthetic credentials, sandboxed services, read-only replicas, and reversible actions. Do not test exploits against live systems, third parties, real user data, or production credentials without explicit authorization and an agreed rules-of-engagement document.

5. Design discriminating tests

Use at least one benign baseline, one capability/control condition, one adversarial condition, and one ablation that distinguishes the favored explanation from a plausible alternative. Add multiple trials for stochastic agents. Measure both safety and task utility.

For monitor audits, estimate false negatives and false positives under realistic base rates; include monitor-independent ground truth when possible; vary information channels; test evaluator awareness, strategic obfuscation, correlated failure, and monitor gaming. For autonomous actions, capture intended plan, observations, tool calls, arguments, results, state changes, approvals, identities, timestamps, policy decisions, and final outcomes.

6. Triangulate and report uncertainty

Require at least two independent evidence modes for high-impact conclusions when feasible: behavioral tests, trace review, human investigation, system logs, static analysis, activation/weight tools, training-data analysis, or incident evidence. Explain conflicts instead of averaging them away.

Use calibrated language. A passed test supports only the tested configuration and threat model. A failed attack is not evidence of absence if capability, access, realism, or search budget was inadequate.

Output contract

Produce the smallest useful set of these sections:

1. Executive decision: decision, tested boundary, major finding, and highest residual risk.
2. Scope and exclusions: versions, access, threat model, cutoff date, and explicit non-claims.
3. Evidence map: comparable source table with evidence grade and independence.
4. System and control map: layers, authority, trust boundaries, and observable evidence.
5. Test plan or results: hypotheses, procedures, baselines, trials, scorers, safety/utility metrics, findings, and artifacts.
6. Control gaps: affected layer, exploit or failure path, severity, likelihood, detectability, evidence strength, and owner.
7. Residual uncertainty: unavailable data, realism and transfer gaps, monitor blind spots, and untested changes.
8. Actions: release gate, containment, monitoring, remediation, re-test trigger, and evidence-retention requirement.
9. Research gaps: only after nearest-prior-art and synonym re-search, using references/research-gaps.md.

Keep claims linked to sources or audit artifacts. Separate direct observation, inference, and recommendation. Include dates and version identifiers in every table that could otherwise become stale.

Minimum quality gate

Do not call work an agent audit unless it has:

• an explicit system boundary and decision;
• a threat model and access statement;
• a versioned evidence register;
• tests or controls tied to specific claims;
• safety and utility measurements, or a reason one is inapplicable;
• scorer and ground-truth limitations;
• stochastic variance or an explicit single-run limitation;
• evidence integrity and retention notes;
• residual uncertainty and re-audit triggers;
• a distinction between first-party claims and independent evidence.

Reject one-number summaries when they conceal severity, scope, base rate, utility cost, or monitor error. Do not infer real-deployment safety from a synthetic model organism, benchmark, or simulated environment without a transfer argument.