Tool Quality Audit

Audit tool integrations for deterministic behavior, error contracts, and logging before agents depend on them. Use when adding or updating tools or MCP servers.

majiayu000 Updated 567 repo stars

File contents

Tool Quality Audit

Overview

Evaluate a tool’s reliability with a checklist and small smoke tests. Emphasize deterministic outputs, clear errors, and stable schemas.

Quick start

  1. Fill templates/tool_audit.json.
  2. Run smoke tests manually or via your harness.
  3. Record findings in results.json.

Core Guidance

  • Prefer deterministic checks before LLM-based grading.
  • Verify error contracts (consistent codes/messages).
  • Validate schemas are stable and documented.
  • Record latency and failure modes.

Resources

  • references/tool-audit-checklist.md: Reliability and contract checklist.
  • templates/tool_audit.json: Audit scaffold for a tool.

Validation

  • Ensure audit file is filled and failures are actionable.

majiayu000/claude-skill-registry-data/tree/main/data/tool-quality-audit commit a12d080f92

Frequently asked questions

npx skillmds@latest add majiayu000/tool-quality-audit