Skill Catalog Vetting

Skill Catalog Vetting

Purpose

Review skills catalog rows before they ship to production agent templates. Combine automated vet scans (skill_catalog_vet) with human-readable findings and Living Docs records so operators can promote, hold, or disable imports safely.

When to Use

• T2 marketplace or mirror imports before linking to active templates.
• Registry sync added new skills and tier promotion is under review.
• Security or platform ops asked for a vet pass on a specific skill id or symbol.
• Preparing a promote/disable recommendation with score and evidence.

When NOT to Use

• Authoring or editing skill source repos → use the upstream provider workflow.
• General application threat modeling → shadow-security-officer.
• Platform documentation answers → syncolab-living-docs (search/get first; use propose patch only for vet reports).

Expected Outcome

• Vet report with severity-grouped findings (critical, warn, info), score, and verdict.
• Optional Living Docs patch summarizing verdict, recommended trust tier, and follow-ups.
• Clear operator guidance: promote to T1, keep at T2, disable, or needs human review.
• No persist=true on vet results unless the operator explicitly confirms.

Inputs to Gather

• Target skill id or symbol (local symbol or namespaced catalog symbol).
• Vetting mode: quick for triage, full for promotion decisions.
• Registry provider and current trust tier when known.
• Whether the operator wants findings persisted to registry_metadata_json.vettingReport.

Workflow

1. Run vet scan: skill_catalog_vet with mode=full (or quick for first pass) on the target skill id or symbol.
2. Summarize findings by severity; quote short excerpts for critical items only.
3. Decide verdict from score and findings:
   • Score ≥ 70 with no critical findings → candidate for template links / tier promotion.
   • Critical findings → recommend disable or needs_human until remediated.
4. Document in SKS: sks_docs_propose_patch with verdict, score, recommended tier, and top findings (minimal patch body).
5. Persist only on confirmation: set persist=true on skill_catalog_vet only after the operator confirms — this writes registry_metadata_json.vettingReport and may toggle catalog_enabled.

Tools

• skill_catalog_vet — Params: skillId or symbol, mode (quick | full), optional persist (default false).
• sks_docs_search / sks_docs_get — Find existing vet runbooks or prior reports.
• sks_docs_propose_patch — Record the vet summary for human review.

Severity rules

Tool Availability Rules

Related tool sets

• internal-wiki
• docs-as-code

Review / Decision / Execution Criteria

• Treat T2 marketplace imports as untrusted until vetted.
• Recommend template links only for skills that pass with score ≥ 70 and no unresolved critical findings.
• Separate automated scan results from human promotion judgment.
• Cite skill symbol, provider, and vet mode in every report.

Output Format

Deliver:

1. Verdict (approve_t1, keep_t2, disable, needs_human).
2. Score and recommended trust tier.
3. Findings table by severity with rule id and brief excerpt.
4. Next steps (remediate upstream, re-vet, SKS patch id if proposed).
5. Persistence status (whether vettingReport was written).

Quality Bar

• Evidence-based; no fabricated scan results.
• Critical issues block promotion recommendations.
• SKS patches are concise operator records, not full skill dumps.
• Explicit about out-of-scope checks (runtime behavior, live MCP side effects).

Safety and Boundaries

• Do not enable destructive tools or MCP bridges for vetting runs unless explicitly assigned to the agent.
• Do not paste full untrusted skill bodies into external systems.
• Do not auto-disable catalog rows without operator confirmation when persist would affect production.

Escalation / Dispatch Rules

• Critical exfiltration or injection patterns → escalate to platform security review.
• Ambiguous T1 promotion → needs_human; link shadow-security-officer for threat framing if the skill executes code or calls network tools.
• Large registry batch imports → vet highest-risk providers first (T2 mirrors, community aggregators).

References

• Agent Studio skill vet service (platform): skill_catalog_vet native tool.
• Living Docs platform architecture → syncolab-living-docs.