Artifact Validator
Audit agent-system artifacts without changing them. Select authority by artifact
type, run the strongest available deterministic checks, and separate observed
facts from unsupported claims.
Overview
This skill is an orchestrator, not another schema authority. In the Tons of
Skills repository it delegates to the repository's canonical validators. In
other projects it applies the open specification floor and the target host's
current contract, clearly labeling checks that could not be reproduced.
When to use
Use for a single file, plugin directory, marketplace, contributor change, or
mixed repository. Use artifact-creator when changes are requested. Use
production-upgrade when the audit must lead through research, implementation,
migration, and release readiness.
Prerequisites
- A target path or repository.
- Read access to project instructions and generated-file ownership rules.
- Network access only when current primary-source verification is necessary.
Never transmit private artifacts to an external validator without approval.
Instructions
Read project instructions, inspect repository status, and determine whether
the target is generated, mirrored, canonical, or host-specific.
Run the offline inventory helper:
python3 scripts/inventory_artifacts.py <target>
Use references/validation-routing.md to
select the owner for each discovered artifact. Do not merge verdicts from
different standards into one unexplained score.
In the Tons of Skills repository:
- use
scripts/validate-skills-schema.py for skill and agent authority;
- use
scripts/validate-mcp-config.mjs and the declared security gates for
MCP configuration;
- run catalog synchronization before judging generated projections;
- keep kernel-shadow results advisory unless governance explicitly changes.
Validate structure, references, declared versus used capabilities, secret
hygiene, path containment, bounded execution, provenance, license, and the
evidence supporting every portability or production claim.
For behavior, require positive, negative, edge, and adversarial cases. A
passing structural score does not prove runtime behavior or safety.
Reproduce automated-review findings independently. Reject or qualify any
claim that lacks a command, primary source, fixture, or observable boundary.
Return a severity-ordered report with exact paths, evidence, authority,
remediation options, and a PASS, FAIL, or NOT-VERIFIED verdict per artifact.
Output
Report the inventory, checks executed, exact command results, blocking findings,
warnings, unverified surfaces, and overall claim ceiling. Do not collapse a
partially tested multi-component plugin into a stronger verdict than its weakest
component.
Error handling
- Missing authoritative tooling: perform the open-spec floor, mark deeper
checks
NOT-VERIFIED, and do not invent a pass.
- Unknown host artifact: identify it as host-specific and request or locate the
current native specification before grading.
- Dirty or generated target: report ownership and drift; do not auto-fix during
an audit.
- Secret-like content: redact the value, identify only the path and field, and
stop any command that could transmit it.
Examples
- A valid
SKILL.md with no behavioral evidence can pass structure while
remaining NOT-VERIFIED for production behavior.
- A Claude plugin with five skills and one unchecked MCP server is capped by the
MCP component's verdict.
- A portable skill claiming Pi, Hermes, or Goose support without registry-backed
receipts is reported as compatible or unverified, not verified-native.
Resources
- Validation routing
scripts/inventory_artifacts.py performs read-only deterministic discovery.
1---2name: artifact-validator3description: Discover and validate Agent Skills, host plugins and subagents, MCP servers and configurations, hooks, and marketplace catalogs against the correct authority. Use when performing pre-merge review, external submission checks, portability audits, security checks, or validation of mixed agent-system repositories. Trigger with "validate this skill", "audit this plugin", "check this agent", "review this MCP config", or "is this marketplace ready".4license: MIT5---6
7# Artifact Validator
8
9Audit agent-system artifacts without changing them. Select authority by artifact
10type, run the strongest available deterministic checks, and separate observed
11facts from unsupported claims.
12
13## Overview
14
15This skill is an orchestrator, not another schema authority. In the Tons of
16Skills repository it delegates to the repository's canonical validators. In
17other projects it applies the open specification floor and the target host's
18current contract, clearly labeling checks that could not be reproduced.
19
20## When to use
21
22Use for a single file, plugin directory, marketplace, contributor change, or
23mixed repository. Use `artifact-creator` when changes are requested. Use
24`production-upgrade` when the audit must lead through research, implementation,
25migration, and release readiness.
26
27## Prerequisites
28
29- A target path or repository.
30- Read access to project instructions and generated-file ownership rules.
31- Network access only when current primary-source verification is necessary.
32 Never transmit private artifacts to an external validator without approval.
33
34## Instructions
35
361. Read project instructions, inspect repository status, and determine whether
37 the target is generated, mirrored, canonical, or host-specific.
382. Run the offline inventory helper:
39
40 ```bash
41 python3 scripts/inventory_artifacts.py <target>
42 ```
43
443. Use [references/validation-routing.md](references/validation-routing.md) to
45 select the owner for each discovered artifact. Do not merge verdicts from
46 different standards into one unexplained score.
474. In the Tons of Skills repository:
48 - use `scripts/validate-skills-schema.py` for skill and agent authority;
49 - use `scripts/validate-mcp-config.mjs` and the declared security gates for
50 MCP configuration;
51 - run catalog synchronization before judging generated projections;
52 - keep kernel-shadow results advisory unless governance explicitly changes.
535. Validate structure, references, declared versus used capabilities, secret
54 hygiene, path containment, bounded execution, provenance, license, and the
55 evidence supporting every portability or production claim.
566. For behavior, require positive, negative, edge, and adversarial cases. A
57 passing structural score does not prove runtime behavior or safety.
587. Reproduce automated-review findings independently. Reject or qualify any
59 claim that lacks a command, primary source, fixture, or observable boundary.
608. Return a severity-ordered report with exact paths, evidence, authority,
61 remediation options, and a PASS, FAIL, or NOT-VERIFIED verdict per artifact.
62
63## Output
64
65Report the inventory, checks executed, exact command results, blocking findings,
66warnings, unverified surfaces, and overall claim ceiling. Do not collapse a
67partially tested multi-component plugin into a stronger verdict than its weakest
68component.
69
70## Error handling
71
72- Missing authoritative tooling: perform the open-spec floor, mark deeper
73 checks `NOT-VERIFIED`, and do not invent a pass.
74- Unknown host artifact: identify it as host-specific and request or locate the
75 current native specification before grading.
76- Dirty or generated target: report ownership and drift; do not auto-fix during
77 an audit.
78- Secret-like content: redact the value, identify only the path and field, and
79 stop any command that could transmit it.
80
81## Examples
82
83- A valid `SKILL.md` with no behavioral evidence can pass structure while
84 remaining `NOT-VERIFIED` for production behavior.
85- A Claude plugin with five skills and one unchecked MCP server is capped by the
86 MCP component's verdict.
87- A portable skill claiming Pi, Hermes, or Goose support without registry-backed
88 receipts is reported as compatible or unverified, not verified-native.
89
90## Resources
91
92- [Validation routing](references/validation-routing.md)
93- `scripts/inventory_artifacts.py` performs read-only deterministic discovery.
gabrielmoreira