12-Factor Agents Compliance Analysis
Reference: 12-Factor Agents
Input Parameters
| Parameter |
Description |
Required |
docs_path |
Path to documentation directory (for existing analyses) |
Optional |
codebase_path |
Root path of the codebase to analyze |
Required |
Analysis Framework
The full per-factor rubric — principle, search patterns, file patterns, compliance criteria (Strong/Partial/Weak), and anti-patterns for each of the 13 factors — lives in references/factors.md. During the Analysis Workflow, read the relevant factor sections there for the search patterns to run and the criteria to score against.
| # |
Factor |
Focus |
| 1 |
Natural Language to Tool Calls |
Schema-validated structured outputs from LLM |
| 2 |
Own Your Prompts |
Prompts as first-class, versioned, templated code |
| 3 |
Own Your Context Window |
Custom formatting of history/state/tool results |
| 4 |
Tools Are Structured Outputs |
Validated JSON triggers deterministic code |
| 5 |
Unify Execution State |
Single state object merging execution + business state |
| 6 |
Launch/Pause/Resume |
APIs to launch, pause anywhere, resume |
| 7 |
Contact Humans with Tools |
Human contact as a structured tool call |
| 8 |
Own Your Control Flow |
Custom routing/retries, not framework defaults |
| 9 |
Compact Errors into Context |
Errors fed back for self-healing + escalation |
| 10 |
Small, Focused Agents |
Narrow responsibility, 3-10 steps each |
| 11 |
Trigger from Anywhere |
CLI/REST/WebSocket/chat/webhook entry points |
| 12 |
Stateless Reducer |
Pure (state, input) -> (state, output) agents |
| 13 |
Pre-fetch Context |
Fetch likely-needed data upfront |
See references/factors.md for the complete rubric for every factor above.
Output Format
Gate order: Do not assign Strong / Partial / Weak or treat recommendations as observed facts until Hard gates (after Analysis Workflow) are satisfied for the factors in scope.
Executive Summary Table
| Factor | Status | Notes |
|--------|--------|-------|
| 1. Natural Language -> Tool Calls | **Strong/Partial/Weak** | [Key finding] |
| 2. Own Your Prompts | **Strong/Partial/Weak** | [Key finding] |
| ... | ... | ... |
| 13. Pre-fetch Context | **Strong/Partial/Weak** | [Key finding] |
**Overall**: X Strong, Y Partial, Z Weak
Per-Factor Analysis
For each factor, provide:
Current Implementation
- Evidence with file:line references
- Code snippets showing patterns
Compliance Level
- Strong/Partial/Weak with justification
Gaps
- What's missing vs. 12-Factor ideal
Recommendations
- Actionable improvements with code examples
Analysis Workflow
Initial Scan
- Run search patterns for all factors
- Identify key files for each factor
- Note any existing compliance documentation
Deep Dive (per factor)
- Read identified files
- Evaluate against compliance criteria
- Document evidence with file paths
Gap Analysis
- Compare current vs. 12-Factor ideal
- Identify anti-patterns present
- Prioritize by impact
Recommendations
- Provide actionable improvements
- Include before/after code examples
- Reference roadmap if exists
Summary
- Compile executive summary table
- Highlight strengths and critical gaps
- Suggest priority order for improvements
Hard gates (evidence before scores)
Run these in order. Do not skip ahead: each Pass is an objective condition you can check (paths on disk, citations present), not internal certainty.
- Scan gate — After the initial scan (workflow step 1), Pass: for every factor (1–13) you have either (a) ≥1 repo-relative path or glob hit to inspect, or (b) a one-line note with rationale (e.g. search command/output, or “no matches — codebase may omit this concern”). Empty hand-waving (“looks fine”) fails this gate.
- Evidence gate (per factor) — Before writing Strong / Partial / Weak for that factor, Pass: “Current Implementation” includes ≥1 citation with file path plus line range or short quoted snippet from
codebase_path, or an explicit no evidence located statement after targeted reads. If evidence is missing after search, default that factor to Weak unless the criterion is clearly N/A (say why).
- Synthesis gate — Executive summary table and per-factor analysis sections, Pass: only after gates 1–2 are satisfied for the factors in scope. Recommendations may name new files or patterns only as proposals; they must not be presented as observed facts without matching citations from step 2.
Quick Reference: Compliance Scoring
| Score |
Meaning |
Action |
| Strong |
Fully implements principle |
Maintain, minor optimizations |
| Partial |
Some implementation, significant gaps |
Planned improvements |
| Weak |
Minimal or no implementation |
High priority for roadmap |
When to Use This Skill
- Evaluating new LLM-powered systems
- Reviewing agent architecture decisions
- Auditing production agentic applications
- Planning improvements to existing agents
- Comparing frameworks or implementations
1---2name: agent-architecture-analysis3description: Use when auditing an agent codebase against the 12-Factor Agents methodology, reviewing LLM-powered system architecture, or assessing agentic app compliance. Triggers on "analyze agent architecture", "12-factor audit", "how compliant is this agent", or "evaluate this LLM app". Also applies when comparing frameworks or planning agent improvements. Not for quick checklists — this performs deep per-factor codebase analysis with file-level evidence.4---5
6# 12-Factor Agents Compliance Analysis
7
8> Reference: [12-Factor Agents](https://github.com/humanlayer/12-factor-agents)
9
10## Input Parameters
11
12| Parameter | Description | Required |
13|-----------|-------------|----------|
14| `docs_path` | Path to documentation directory (for existing analyses) | Optional |
15| `codebase_path` | Root path of the codebase to analyze | Required |
16
17## Analysis Framework
18
19The full per-factor rubric — principle, search patterns, file patterns, compliance criteria (Strong/Partial/Weak), and anti-patterns for each of the 13 factors — lives in [references/factors.md](references/factors.md). During the [Analysis Workflow](#analysis-workflow), read the relevant factor sections there for the search patterns to run and the criteria to score against.
20
21| # | Factor | Focus |
22|---|--------|-------|
23| 1 | Natural Language to Tool Calls | Schema-validated structured outputs from LLM |
24| 2 | Own Your Prompts | Prompts as first-class, versioned, templated code |
25| 3 | Own Your Context Window | Custom formatting of history/state/tool results |
26| 4 | Tools Are Structured Outputs | Validated JSON triggers deterministic code |
27| 5 | Unify Execution State | Single state object merging execution + business state |
28| 6 | Launch/Pause/Resume | APIs to launch, pause anywhere, resume |
29| 7 | Contact Humans with Tools | Human contact as a structured tool call |
30| 8 | Own Your Control Flow | Custom routing/retries, not framework defaults |
31| 9 | Compact Errors into Context | Errors fed back for self-healing + escalation |
32| 10 | Small, Focused Agents | Narrow responsibility, 3-10 steps each |
33| 11 | Trigger from Anywhere | CLI/REST/WebSocket/chat/webhook entry points |
34| 12 | Stateless Reducer | Pure `(state, input) -> (state, output)` agents |
35| 13 | Pre-fetch Context | Fetch likely-needed data upfront |
36
37See [references/factors.md](references/factors.md) for the complete rubric for every factor above.
38
39---
40
41## Output Format
42
43**Gate order:** Do not assign Strong / Partial / Weak or treat recommendations as observed facts until **Hard gates** (after [Analysis Workflow](#analysis-workflow)) are satisfied for the factors in scope.
44
45### Executive Summary Table
46
47```markdown
48| Factor | Status | Notes |
49|--------|--------|-------|
50| 1. Natural Language -> Tool Calls | **Strong/Partial/Weak** | [Key finding] |
51| 2. Own Your Prompts | **Strong/Partial/Weak** | [Key finding] |
52| ... | ... | ... |
53| 13. Pre-fetch Context | **Strong/Partial/Weak** | [Key finding] |
54
55**Overall**: X Strong, Y Partial, Z Weak
56```
57
58### Per-Factor Analysis
59
60For each factor, provide:
61
621. **Current Implementation**
63 - Evidence with file:line references
64 - Code snippets showing patterns
65
662. **Compliance Level**
67 - Strong/Partial/Weak with justification
68
693. **Gaps**
70 - What's missing vs. 12-Factor ideal
71
724. **Recommendations**
73 - Actionable improvements with code examples
74
75---
76
77## Analysis Workflow
78
791. **Initial Scan**
80 - Run search patterns for all factors
81 - Identify key files for each factor
82 - Note any existing compliance documentation
83
842. **Deep Dive** (per factor)
85 - Read identified files
86 - Evaluate against compliance criteria
87 - Document evidence with file paths
88
893. **Gap Analysis**
90 - Compare current vs. 12-Factor ideal
91 - Identify anti-patterns present
92 - Prioritize by impact
93
944. **Recommendations**
95 - Provide actionable improvements
96 - Include before/after code examples
97 - Reference roadmap if exists
98
995. **Summary**
100 - Compile executive summary table
101 - Highlight strengths and critical gaps
102 - Suggest priority order for improvements
103
104---
105
106## Hard gates (evidence before scores)
107
108Run these in order. Do not skip ahead: each **Pass** is an objective condition you can check (paths on disk, citations present), not internal certainty.
109
1101. **Scan gate** — After the initial scan (workflow step 1), **Pass:** for every factor (1–13) you have either (a) ≥1 repo-relative path or glob hit to inspect, or (b) a one-line note with rationale (e.g. search command/output, or “no matches — codebase may omit this concern”). Empty hand-waving (“looks fine”) fails this gate.
1112. **Evidence gate (per factor)** — Before writing Strong / Partial / Weak for that factor, **Pass:** “Current Implementation” includes ≥1 citation with **file path** plus **line range or short quoted snippet** from `codebase_path`, or an explicit **no evidence located** statement after targeted reads. If evidence is missing after search, default that factor to **Weak** unless the criterion is clearly N/A (say why).
1123. **Synthesis gate** — Executive summary table and per-factor analysis sections, **Pass:** only after gates 1–2 are satisfied for the factors in scope. Recommendations may name new files or patterns only as proposals; they must not be presented as observed facts without matching citations from step 2.
113
114---
115
116## Quick Reference: Compliance Scoring
117
118| Score | Meaning | Action |
119|-------|---------|--------|
120| **Strong** | Fully implements principle | Maintain, minor optimizations |
121| **Partial** | Some implementation, significant gaps | Planned improvements |
122| **Weak** | Minimal or no implementation | High priority for roadmap |
123
124## When to Use This Skill
125
126- Evaluating new LLM-powered systems
127- Reviewing agent architecture decisions
128- Auditing production agentic applications
129- Planning improvements to existing agents
130- Comparing frameworks or implementations