JusttYashh Skill Audit
Test whether a compiled Skill still matches how work actually happens.
Read schemas in ../soulcatcher/reference.md.
Input
- Compiled skill in
learned-skills/{skill-name}/(SKILL.md + evidence.md) - Recent BehavioralTraces (3+ sessions since compile date, or user-specified window)
Process
- Read compiled SKILL.md procedure steps
- Read evidence.md baseline (original sessions, sequence, confidence)
- Observe recent sessions → new BehavioralTraces
- Compare recent sequences to compiled steps
- Score alignment per core step (% of recent sessions where step appeared)
- Issue verdict
Alignment Scoring
| Core step present in session | Score |
|---|---|
| Yes, same position | 1.0 |
| Yes, different position | 0.5 |
| Skipped | 0.0 |
Alignment = average across core steps and recent sessions × 100
Verdicts
| Verdict | Condition | Action |
|---|---|---|
still_valid |
Alignment ≥ 80% | No change |
drift_detected |
Alignment 50–79% | Propose SKILL.md diff or re-run pipeline |
obsolete |
Alignment < 50% | Recommend deprecating or full re-learn |
Output Format
# Skill Audit: {skill-name}
**Verdict:** still_valid | drift_detected | obsolete
**Alignment:** {N}%
**Sessions audited:** {count}
**Compile date:** {from evidence.md}
## Step alignment
| Step | Recent presence |
|------|-----------------|
| Generate candidates | 5/5 sessions |
| Search counterexamples | 3/5 sessions ⚠ |
| ... | ... |
## Drift details
{What changed — e.g. user now skips counterexample search under time pressure}
## Recommendation
{Keep skill | Patch step 4 as optional | Re-run Soulcatcher on recent sessions}
Cadence
Suggest audit when:
- User requests it
- 10+ new relevant sessions since compile
- Major role or project shift