Agent Governance

Agent governance patterns

Add governance controls to AI agent systems so tool use, content handling, delegation, rate, and accountability stay inside explicit safety and trust boundaries.

When to invoke

• "Add governance to an agent that calls external tools."
• "Implement policy-based access control for agent tools."
• "Detect dangerous prompts before tool execution."
• "Create trust scoring for a multi-agent workflow."
• "Add audit trails, rate limits, content filters, or tool restrictions."

Governance flow

User Request → Intent Classification → Policy Check → Tool Execution → Audit Log
                     ↓                      ↓               ↓
              Threat Detection         Allow/Deny      Trust Update

Apply controls before tool execution, update trust after outcomes, and write audit records for both allowed and denied actions.

Governance levels

Control patterns

Implementation checklist

## Agent Governance Implementation Checklist

### Setup
- [ ] Define governance policy (allowed tools, blocked patterns, rate limits)
- [ ] Choose governance level (open/standard/strict/locked)
- [ ] Set up audit trail storage

### Implementation
- [ ] Add @govern decorator to all tool functions
- [ ] Add intent classification to user input processing
- [ ] Implement trust scoring for multi-agent interactions
- [ ] Wire up audit trail export

### Validation
- [ ] Test that blocked tools are properly denied
- [ ] Test that content filters catch sensitive patterns
- [ ] Test rate limiting behavior
- [ ] Verify audit trail captures all events
- [ ] Test policy composition (most-restrictive-wins)

Framework integration cues

Read references/pattern-implementations.md before writing code for policy objects, classifiers, decorators, trust scoring, audit trails, or framework-specific examples.

Criteria

• The agent's external tools, APIs, databases, file systems, and shell access are inventoried.
• Each tool has an allow, deny, approval, and audit rule.
• Intent classification runs before side-effecting tools.
• Dangerous prompt, sensitive data, and content filter rules are explicit.
• Trust scores have update and decay rules.
• Audit records are append-only and include actor, intent, tool, decision, timestamp, and outcome.
• Governance errors fail closed.

Gotchas

• Do not hardcode policy in tool bodies: policy must be reviewable and changeable outside business logic.
• Do not rely on audit-only for production sensitive operations: audit after harm is not prevention.
• Do not let dynamic tool discovery bypass allowlists: discovered tools still need explicit policy.
• Do not delete audit entries: corrections should append compensating records.

Progressive disclosure and bundled resources

• references/pattern-implementations.md: concrete implementations for policy, classifier, decorator, trust scoring, audit trail, and framework integration.

Environment shorthand

Use open governance for dev/testing; increase controls before production.

Output template

## Agent governance plan

**Status:** ready | implemented | blocked
**Governance level:** open | standard | strict | locked
**Agent/framework:** <name>

| Control | Decision | Evidence or implementation |
| --- | --- | --- |
| Tool allowlist | <rule> | <policy path or code seam> |
| Intent classifier | <rule> | <classifier/prompt/model> |
| Human approval | <rule> | <approval path> |
| Audit trail | <rule> | <storage/export> |
| Trust scoring | <rule> | <update/decay formula> |

### Validation
- Blocked tool denied: pass | fail
- Content filter tested: pass | fail
- Rate limit tested: pass | fail
- Audit event captured: pass | fail

Quality gate

• Governance level is selected based on risk: open, standard, strict, or locked.
• Policies are configuration or data, not scattered hardcoded conditionals.
• Most-restrictive-wins behavior is implemented or specified.
• Side-effecting tool calls require pre-flight intent and policy checks.
• Audit logging is append-only and records allowed and denied actions.
• Trust scoring includes decay over time.
• Fail-closed behavior is tested for policy, classifier, and audit failures.

References

• Agent Governance Toolkit
• AgentMesh Integrations
• OWASP Top 10 for LLM Applications