Secure AI agent coding
Keep an agent's blast radius small and its actions auditable. Scale controls to the real trust boundaries, data sensitivity and effects: a read-only public-data assistant, a disposable no-network sandbox and a production payment agent need different controls. Never demand every control everywhere.
Positions to hold
- Authorization, validation and limits are enforced outside the model. A prompt that says "do not delete data" is not a control, and a second model used as validator is another injectable surface; use deterministic checks (schemas, allowlists, permissions, resource limits), with classifiers only as a supplement.
- Everything not generated by a trusted server-side component is untrusted input: user text, web pages, documents, email, retrieved chunks, tool output, other models' output. Keep instructions and data in separate fields, delimit retrieved content, record its provenance.
- "Authenticated user" is not "authorised agent action". Check each action server-side with credentials scoped to the agent, never a borrowed admin account. UI-only guardrails do not count.
- Classify each action by its actual effect, not by whether a tool or code execution is involved. Reads and reversible internal writes are low; consequential communications, record changes, money, destructive or privileged actions, access grants and highly sensitive data are high. High-impact actions need approval shown before execution with the exact action, target and data, or an accepted equivalent control. Do not demand repeated human approval for work already authorised inside an enforced sandbox, and do not let sandbox authority carry over to production databases or outgoing mail.
- Broad tools (raw shell, wildcard file/DB/API access) become narrow named actions with validated arguments, or an enforced isolated sandbox with resource limits. Broad host access plus a warning is not a sandbox.
- Validate model output before it reaches SQL, shell,
eval, HTML, file paths, APIs or workflow engines.
- Keep sensitive data out of prompts and logs unless the model needs it; redact, tokenise or use synthetic data. Logs and traces become a data store. Consent is per processing purpose, not a blanket terms checkbox.
- Agent memory and workflow state are initialised explicitly per user and task; nothing leaks across users or retries.
- Model, prompt, framework or retrieval changes alter behaviour without a code diff. Pin behaviour-sensitive versions where possible and rerun safety and structured-output tests after changes. Treat safety warnings as errors until triaged.
- Document any control you skip, who accepted the risk, and when it expires.
Reviewing
Lead with findings: severity, evidence (file, endpoint, config), concrete risk, specific fix; then open questions and residual risk. Blocker means exposure of sensitive data, unapproved high-impact action, authorization bypass, or continuing after partial failure; High means prompt-only controls, broad permissions, unvalidated output or missing rollback for consequential actions.
scripts/scan_patterns.py <path> [--json] [--fail-on high] greps for exec/eval or SQL on model output, shell=True, wildcard tools, disabled safety filters, prompt concatenation of untrusted input, hardcoded provider keys and approval bypass flags. Use it to find leads in unfamiliar code; a clean result proves nothing.
Read references/review-workflow.md for the full review phases (surface, trust boundaries, data, capabilities, action tiers). Read references/threat-model.md for RAG, tool-misuse and data-flow scenarios when threat modelling. Read references/implementation-patterns.md for code patterns: prompt boundary, structured-output gate, action registry, reversible-first, transaction boundary, RAG quarantine, rate and budget limits, redacted observability, safety regression tests.
When bundled advice conflicts with a provider or framework contract, check version-matched docs and OWASP, NIST or MITRE guidance, and report the discrepancy instead of silently editing this skill.
1---2name: secure-ai-agent-coding3description: "Reviews and hardens AI agents, LLM apps, RAG, tool-calling and coding agents: prompt injection, tool permissions, approval gates, model-output sinks, AI data handling, safety regression tests. Use where a model's text can trigger actions or touch data. Not for non-AI appsec."4metadata:5 source: https://skillmd.com/skills/jpcaparas/secure-ai-agent-coding6---78# Secure AI agent coding910Keep an agent's blast radius small and its actions auditable. Scale controls to the real trust boundaries, data sensitivity and effects: a read-only public-data assistant, a disposable no-network sandbox and a production payment agent need different controls. Never demand every control everywhere.1112## Positions to hold1314- Authorization, validation and limits are enforced outside the model. A prompt that says "do not delete data" is not a control, and a second model used as validator is another injectable surface; use deterministic checks (schemas, allowlists, permissions, resource limits), with classifiers only as a supplement.15- Everything not generated by a trusted server-side component is untrusted input: user text, web pages, documents, email, retrieved chunks, tool output, other models' output. Keep instructions and data in separate fields, delimit retrieved content, record its provenance.16- "Authenticated user" is not "authorised agent action". Check each action server-side with credentials scoped to the agent, never a borrowed admin account. UI-only guardrails do not count.17- Classify each action by its actual effect, not by whether a tool or code execution is involved. Reads and reversible internal writes are low; consequential communications, record changes, money, destructive or privileged actions, access grants and highly sensitive data are high. High-impact actions need approval shown before execution with the exact action, target and data, or an accepted equivalent control. Do not demand repeated human approval for work already authorised inside an enforced sandbox, and do not let sandbox authority carry over to production databases or outgoing mail.18- Broad tools (raw shell, wildcard file/DB/API access) become narrow named actions with validated arguments, or an enforced isolated sandbox with resource limits. Broad host access plus a warning is not a sandbox.19- Validate model output before it reaches SQL, shell, `eval`, HTML, file paths, APIs or workflow engines.20- Keep sensitive data out of prompts and logs unless the model needs it; redact, tokenise or use synthetic data. Logs and traces become a data store. Consent is per processing purpose, not a blanket terms checkbox.21- Agent memory and workflow state are initialised explicitly per user and task; nothing leaks across users or retries.22- Model, prompt, framework or retrieval changes alter behaviour without a code diff. Pin behaviour-sensitive versions where possible and rerun safety and structured-output tests after changes. Treat safety warnings as errors until triaged.23- Document any control you skip, who accepted the risk, and when it expires.2425## Reviewing2627Lead with findings: severity, evidence (file, endpoint, config), concrete risk, specific fix; then open questions and residual risk. Blocker means exposure of sensitive data, unapproved high-impact action, authorization bypass, or continuing after partial failure; High means prompt-only controls, broad permissions, unvalidated output or missing rollback for consequential actions.2829`scripts/scan_patterns.py <path> [--json] [--fail-on high]` greps for exec/eval or SQL on model output, `shell=True`, wildcard tools, disabled safety filters, prompt concatenation of untrusted input, hardcoded provider keys and approval bypass flags. Use it to find leads in unfamiliar code; a clean result proves nothing.3031Read `references/review-workflow.md` for the full review phases (surface, trust boundaries, data, capabilities, action tiers). Read `references/threat-model.md` for RAG, tool-misuse and data-flow scenarios when threat modelling. Read `references/implementation-patterns.md` for code patterns: prompt boundary, structured-output gate, action registry, reversible-first, transaction boundary, RAG quarantine, rate and budget limits, redacted observability, safety regression tests.3233When bundled advice conflicts with a provider or framework contract, check version-matched docs and OWASP, NIST or MITRE guidance, and report the discrepancy instead of silently editing this skill.