Quick Start Guide

Quick Start Guide

Installation

Using uv (Recommended)

# Install uv if you haven't already
curl -LsSf https://astral.sh/uv/install.sh | sh

# Clone and setup
git clone https://github.com/cisco-ai-defense/skill-scanner
cd skill-scanner

# Install all dependencies
uv sync --all-extras

Using pip

# Install the package
pip install cisco-ai-skill-scanner[all]

Basic Usage

Environment Setup (Optional)

# For LLM analyzer and Meta-analyzer
export SKILL_SCANNER_LLM_API_KEY="your_api_key"
export SKILL_SCANNER_LLM_MODEL="claude-3-5-sonnet-20241022"

# For VirusTotal binary scanning
export VIRUSTOTAL_API_KEY="your_virustotal_api_key"

# For Cisco AI Defense
export AI_DEFENSE_API_KEY="your_aidefense_api_key"

Scan a Single Skill

# From source (with uv)
uv run skill-scanner scan evals/skills/safe-skills/simple-math

# Installed package
skill-scanner scan evals/skills/safe-skills/simple-math

By default, scan runs the core analyzers: static + bytecode + pipeline.

Scan Multiple Skills

# Scan all skills in a directory
skill-scanner scan-all evals/skills --format table

# Recursive scan with detailed markdown report
skill-scanner scan-all evals/skills --format markdown --detailed --output report.md

Demo Results

The project includes test skills in evals/skills/ for evaluation and testing:

[OK] simple-math (SAFE)

$ skill-scanner scan evals/skills/safe-skills/simple-math
============================================================
Skill: simple-math
============================================================
Status: [OK] SAFE
Max Severity: SAFE
Total Findings: 0
Scan Duration: 0.27s

[FAIL] multi-file-exfiltration (CRITICAL)

$ skill-scanner scan evals/skills/behavioral-analysis/multi-file-exfiltration --use-behavioral
============================================================
Skill: config-analyzer
============================================================
Status: [FAIL] ISSUES FOUND
Max Severity: CRITICAL
Total Findings: 11
Scan Duration: 0.42s

Findings Summary:
  Critical: 3
  High:     3
  Medium:   4
  Low:      1

Detected Threats:

• Data exfiltration (HTTP POST to external server)
• Reading sensitive files (~/.aws/credentials)
• Environment variable theft (API_KEY, SECRET_TOKEN)
• Command injection (eval on user input)
• Base64 encoding + network exfiltration pattern

Useful Commands

# List available analyzers
skill-scanner list-analyzers

# Validate rule signatures
skill-scanner validate-rules

# Get help
skill-scanner --help
skill-scanner scan --help

Output Formats

JSON (for CI/CD)

skill-scanner scan /path/to/skill --format json --output results.json

SARIF (for GitHub Code Scanning)

skill-scanner scan /path/to/skill --format sarif --output results.sarif

Markdown (human-readable report)

skill-scanner scan /path/to/skill --format markdown --detailed --output report.md

Table (terminal-friendly)

skill-scanner scan-all evals/skills --format table

Advanced Features

Scan Policies

Use built-in presets or a custom policy to tune detection sensitivity:

# Use a stricter preset
skill-scanner scan /path/to/skill --policy strict

# Use a more permissive preset
skill-scanner scan /path/to/skill --policy permissive

# Generate a custom policy YAML to edit
skill-scanner generate-policy -o my_policy.yaml

# Interactive policy configurator (TUI)
skill-scanner configure-policy

See Scan Policy Guide for full details.

Enable All Analyzers

skill-scanner scan /path/to/skill \
  --use-behavioral \
  --use-llm \
  --use-trigger \
  --use-aidefense \
  --use-virustotal

LLM provider note: --llm-provider currently accepts anthropic or openai. For Bedrock, Vertex, Azure, Gemini, and other LiteLLM backends, set provider-specific model strings and environment variables (see docs/llm-analyzer.md).

Cross-Skill Analysis

skill-scanner scan-all /path/to/skills --check-overlap

Pre-commit Hook

cp scripts/pre-commit-hook.sh .git/hooks/pre-commit
chmod +x .git/hooks/pre-commit

Next Steps

1. Review the documentation:

   • README.md - Project overview
   • docs/architecture.md - System design
   • docs/threat-taxonomy.md - All threat categories
   • docs/scan-policy.md - Custom policies and tuning

2. Try scanning your own skills:

   skill-scanner scan /path/to/your/skill
   

3. Integrate with CI/CD:

   skill-scanner scan-all ./skills --fail-on-findings
   # Exit code 1 if critical/high issues found
   

Troubleshooting

UV not found

Install UV:

curl -LsSf https://astral.sh/uv/install.sh | sh

Module not found errors

Sync dependencies:

uv sync --all-extras

Permission errors

UV manages its own virtual environment - no need for manual venv activation.