aibot88 Graph Audit
Coverage
- Schema conformance: checking that every
skills/<name>/SKILL.mdvalidates againstschemas/skill.schema.jsonwithout errors - Manifest sync: verifying that
examples/skills.manifest.sample.jsonmatches the output ofscripts/generate-manifest.jsrun against the current skills - Relation integrity: confirming that every target named under
relations.*corresponds to a real sibling skill directory and uses the right predicate semantics (related/broader/boundary/disjoint_with/verify_with/depends_on) - Eval artifact coherence: ensuring that
eval_artifacts: presentis backed by a real eval artifact underexamples/evals/that names the skill in itsskill_namefield - Grounding presence: confirming that every
scope: codebaseskill has a fully populatedgroundingblock withdomain_object,grounding_mode,truth_sources,failure_modes, andevidence_priority - Name-directory parity: checking that a skill's
namefield matches the name of the parent directory (required for SKILL.md compatibility)
Philosophy
Skill graphs fail silently. A broken relation or a drifted enum value does not crash the agent — it just makes retrieval subtly wrong, and subtly-wrong retrieval is worse than a crash because nothing tells you to look. The audit's job is to turn every silent bug into a loud one before the graph accumulates enough drift that agents can no longer trust its edges.
Key Files
| File | Line range | Purpose |
|---|---|---|
schemas/skill.schema.json |
whole file | Enforces the frontmatter schema for every SKILL.md |
schemas/manifest.schema.json |
whole file | Enforces the compiled manifest shape |
docs/skill-metadata-protocol.md |
§§ Archetype, Requiredness, Schema Versioning | Source of truth for field semantics and the archetype section map |
scripts/skill-lint.js |
91–114 (AUTHORED_FIELDS_MUST_FLOW), 149–202 (checkSchemaParity), 175–250 (validateAgainstSchema) |
The canonical audit runner. Implements the six dimensions listed in Coverage plus five more: parent-directory-matches-name, cross-schema parity, sample-manifest conformance, generator parity, and routing-quality rules. See README § Validation for the full eleven-check list. |
scripts/lib/alias-contract.js |
whole file | Shared v3.1 alias-parity guard used by lint and manifest generation so preferred/legacy fields cannot diverge silently |
scripts/check-protocol-consistency.js |
C1–C7 checks | Cross-artifact protocol checker. Complementary to skill-lint.js — lint validates per-skill correctness; this validates that the protocol documents themselves remain consistent with the schemas. |
examples/skills.manifest.sample.json |
whole file | Generator-produced sample; lint fails if this drifts from generate-manifest.js output |
Evals
This skill ships a comprehension-eval artifact at examples/evals/graph-audit.json covering all six audit dimensions listed under Coverage. The Verification checklist below is the deterministic per-file audit gate; the eval file is how this skill's concept comprehension is graded by scripts/skill-audit.js --graded.
Verification
Run the lint script to execute all audit checks:
node scripts/skill-lint.js
For a single skill:
node scripts/skill-lint.js skills/<name>
For the full audit including the skill-metadata-template:
node scripts/skill-lint.js --include-template
Exit code 0 means all checks passed. Exit code 1 means at least one check failed; each failure identifies the specific file and check.
- All SKILL.md files pass schema validation
- Manifest sample matches generator output
- All relation targets exist as real sibling skill directories
- All
eval_artifacts: presentskills have a matching eval artifact - All
scope: codebaseskills have a completegroundingblock - Every skill's
namefield matches its parent directory name (SKILL.md compatibility)
Do NOT Use When
| Use instead | When |
|---|---|
documentation |
The task is authoring or restructuring skill prose, not auditing metadata |
refactor |
The task is restructuring skill body sections while keeping the contract stable |
debugging |
The task is chasing a runtime failure in an agent, not validating graph metadata |