jmagly soul-validate
Validate a SOUL.md file against community best practices and quality criteria.
Triggers
Alternate expressions and non-obvious activations (primary phrases are matched automatically from the skill description):
- "soul score" → soul quality metrics
- "check the soul" → soul validation
Behavior
When triggered, this skill performs a comprehensive quality assessment of a SOUL.md file, checking for completeness, specificity, and effectiveness.
Validation Process
- Locate SOUL.md — check
./SOUL.md,./.aiwg/SOUL.md, or path from user - Parse sections — identify which recommended sections are present
- Assess each section for quality criteria
- Report results with specific, actionable feedback
Section Checklist
| Section | Required | Quality Criteria |
|---|---|---|
| Who I Am | Yes | Specific background, not generic. Mentions concrete experience. |
| Worldview | Yes | Beliefs specific enough to be falsifiable. Not platitudes. |
| Opinions | Yes | Organized by domain. Each opinion could be disagreed with. |
| Standards | Recommended | Concrete bar for completion. Names dangling threads this persona never leaves and how complete-vs-fast tension resolves. Not "I value quality." |
| Vocabulary | Recommended | Actual terms with personal definitions, not categories. |
| Boundaries | Recommended | Concrete refusals, not vague ethical statements. |
| Current Focus | Optional | Active projects, current thinking. Dated or contextual. |
| Influences | Optional | Specific people/works with what was taken from each. |
| Tensions | Optional | Genuine contradictions — shows self-awareness. |
| Pet Peeves | Optional | Specific triggers, not generic annoyances. |
| Interests | Optional | Deep interests that inform cross-domain thinking. |
Backwards compatibility: Souls that predate the Standards section validate with a warning, not an error. Existing souls remain functional; /soul-enhance proposes Standards content. New souls created via /soul-create should populate the section.
Quality Tests
1. Prediction Test (Critical)
"Could someone reading this SOUL.md predict the agent's takes on new, unstated topics?"
- Pass: Worldview + opinions are specific enough to interpolate
- Fail: Too vague — anyone could have these opinions
2. Specificity Test
Check for vague language patterns that weaken the soul:
| Vague (Fail) | Specific (Pass) |
|---|---|
| "I have nuanced views on X" | "I think X is overrated because Y" |
| "I value quality" | "I'll delay a release to fix a flaky test" |
| "I aim for excellence" | "Done = no dangling threads, real fix not workaround" |
| "I believe in completeness" | "I never ship a fix without a regression test for it" |
| "I'm interested in technology" | "I've spent 10 years on distributed systems" |
| "I believe in best practices" | "Most 'best practices' are cargo cult" |
Flag any sentence that could apply to anyone.
3. Context Budget Test
- Under 5K tokens (~3,750 words): Pass
- 5K-8K tokens: Warning — consider trimming
- Over 8K tokens: Fail — too large for context budget
4. Anti-Pattern Detection
Flag common anti-patterns:
| Anti-Pattern | Example | Fix |
|---|---|---|
| Generic positivity | "I'm passionate about helping" | Replace with specific beliefs |
| Exhaustive rules | 50+ boundary statements | Reduce to 5-10 core boundaries |
| No contradictions | Suspiciously coherent persona | Add 2-3 genuine tensions |
| Category-level vocabulary | "technical terms" | List actual terms with definitions |
| Missing opinions | Worldview without takes | Add domain-specific opinions |
5. Companion File Check
Check for recommended companion files:
| File | Purpose | Status |
|---|---|---|
| STYLE.md | Writing patterns | Check if exists |
| examples/good-outputs.md | Calibration examples | Check if exists |
| examples/bad-outputs.md | Anti-pattern examples | Check if exists |
Output Format
Soul Validation Report
=======================
File: ./SOUL.md (~2,847 tokens)
Section Completeness
---------------------
✓ Who I Am Present, specific
✓ Worldview Present, 4 falsifiable beliefs
✓ Opinions Present, organized by 3 domains
⚠ Standards Missing — what's this persona's bar for "done"? (warning, not error)
✓ Vocabulary Present, 12 terms defined
✗ Boundaries Missing — what will the agent refuse?
✓ Current Focus Present
✓ Influences Present, 5 named with takeaways
✗ Tensions Missing — what contradictions exist?
✓ Pet Peeves Present, 4 specific triggers
Quality Tests
--------------
✓ Prediction test Opinions are interpolatable
⚠ Specificity test 2 vague statements found:
Line 14: "I value clean code" → too generic, what specifically?
Line 31: "I have experience with many frameworks" → which ones?
✓ Context budget ~2,847 tokens (under 5K limit)
✓ Anti-patterns None detected
Companion Files
----------------
✗ STYLE.md Not found
✗ examples/good-outputs.md Not found
✗ examples/bad-outputs.md Not found
Score: 7/10
Recommendations
----------------
1. Add a Boundaries section — define 5-10 concrete refusals
2. Add a Tensions section — list 2-3 genuine contradictions
3. Fix vague statements on lines 14 and 31
4. Consider creating calibration examples (good-outputs.md)
5. Consider creating a STYLE.md companion for writing patterns
Run /soul-enhance to auto-improve these issues.
Scoring
| Score | Meaning |
|---|---|
| 9-10 | Production-ready. Distinctive, interpolatable, well-bounded. |
| 7-8 | Good foundation. Missing sections or minor vagueness. |
| 5-6 | Needs work. Multiple missing sections or pervasive vagueness. |
| 3-4 | Weak. Mostly generic. Fails prediction test. |
| 1-2 | Placeholder. Not functional as a soul file. |
Integration
With soul-create
After /soul-create, automatically offer validation:
SOUL.md created. Run /soul-validate to check quality.
With soul-enable
Before enabling, recommend validation if score < 7:
Warning: SOUL.md scores 5/10 on quality check.
Consider running /soul-validate and /soul-enhance before enabling.
Examples
# Validate project soul
/soul-validate
# Validate specific file
/soul-validate .aiwg/SOUL.md
# Validate agent soul
/soul-validate .claude/agents/test-engineer.soul.md
References
- @$AIWG_ROOT/agentic/code/addons/aiwg-utils/skills/soul-create/SKILL.md — Soul creation
- @$AIWG_ROOT/agentic/code/addons/aiwg-utils/commands/soul-enable.md — Soul enforcement
- @$AIWG_ROOT/docs/soul-md-guide.md — SOUL.md integration guide
- #437 — SOUL.md compatibility issue