AI Defense Analyzer

AI Defense Analyzer

Overview

The AI Defense Analyzer integrates with Cisco AI Defense API to provide enterprise-grade security scanning for Agent Skills. It analyzes prompts, instructions, markdown content, and code files for threats including prompt injection, data exfiltration, and malicious patterns.

Features

• Prompt Injection Detection: Identifies attempts to manipulate AI behavior through crafted inputs
• Data Exfiltration Detection: Flags patterns that could leak sensitive information
• Tool Poisoning Detection: Detects malicious tool descriptions and parameters
• Code Security Analysis: Scans Python and other code files for security vulnerabilities
• Real-time API Analysis: Leverages Cisco AI Defense cloud infrastructure
• Retry Logic: Built-in exponential backoff for rate limit handling

Configuration

API Key Setup

Set your API key via environment variable or pass it directly:

# Environment variable (recommended)
export AI_DEFENSE_API_KEY="your_api_key"

# Or via .env file
echo "AI_DEFENSE_API_KEY=your_key" >> .env

Configuration Options

Default Enabled Rules:

• Prompt Injection
• Harassment
• Hate Speech
• Profanity
• Sexual Content & Exploitation
• Social Division & Polarization
• Violence & Public Safety Threats
• Code Detection (excluded for code files, included for prompts/markdown)

Important: The "Code Detection" rule is automatically excluded when analyzing actual code files (Python scripts) to avoid false positives, since skills legitimately contain code. Code Detection is still used for prompts, markdown, and manifest content where malicious code injection would be a security concern.

Usage

Command Line

# Enable AI Defense analyzer
skill-scanner scan /path/to/skill --use-aidefense

# Provide API key directly
skill-scanner scan /path/to/skill --use-aidefense --aidefense-api-key your_key

# Combine with other analyzers
skill-scanner scan /path/to/skill --use-behavioral --use-llm --use-aidefense

# Scan multiple skills
skill-scanner scan-all /path/to/skills --recursive --use-aidefense

Python API

from skill_scanner.core.analyzers import AIDefenseAnalyzer
from skill_scanner.core.loader import load_skill

# Initialize analyzer with default rules
analyzer = AIDefenseAnalyzer(
    api_key="your_api_key",  # Or set AI_DEFENSE_API_KEY env var
    timeout=60,
    max_retries=3
)

# Initialize with custom rules
from skill_scanner.core.analyzers.aidefense_analyzer import DEFAULT_ENABLED_RULES

custom_rules = [
    {"rule_name": "Prompt Injection"},
    {"rule_name": "Code Detection"},  # Will be excluded for code files automatically
]

analyzer = AIDefenseAnalyzer(
    api_key="your_api_key",
    enabled_rules=custom_rules,
    include_rules=True  # Set to False if API key has pre-configured rules
)

# Synchronous analysis
skill = load_skill("/path/to/skill")
findings = analyzer.analyze(skill)

# Async analysis (preferred for batch operations)
import asyncio

async def scan_skill():
    findings = await analyzer.analyze_async(skill)
    return findings

findings = asyncio.run(scan_skill())

Integration with Scanner

from skill_scanner import SkillScanner
from skill_scanner.core.analyzers import StaticAnalyzer, AIDefenseAnalyzer

# Combine analyzers
analyzers = [
    StaticAnalyzer(),
    AIDefenseAnalyzer(api_key="your_key"),
]

scanner = SkillScanner(analyzers=analyzers)
result = scanner.scan_skill("/path/to/skill")

How It Works

Analysis Pipeline

1. Content Extraction: Extracts content from SKILL.md, manifest, markdown files, and code files
2. API Request: Sends content to Cisco AI Defense /inspect/chat endpoint
3. Response Processing: Parses classifications, rules, and actions from API response
4. Finding Generation: Converts API results to standardized Finding objects

Content Types Analyzed

API Response Mapping

The analyzer maps Cisco AI Defense classifications to internal severity levels:

Error Handling

The analyzer handles errors gracefully:

• Rate Limits (429): Automatic retry with exponential backoff
• Authentication (401/403): Clear error message for invalid API keys
• Timeouts: Configurable timeout with retry attempts
• Network Errors: Logged errors, partial results returned

Integration with Other Analyzers

For comprehensive coverage, combine AI Defense with other analyzers:

# Maximum coverage
skill-scanner scan /path/to/skill \
    --use-behavioral \
    --use-llm \
    --use-aidefense \
    --use-virustotal

Best Practices

1. Set API key via environment: Avoid hardcoding keys in scripts
2. Use async for batch scans: Improves throughput for multiple skills
3. Combine with static analysis: AI Defense complements pattern-based detection
4. Monitor API usage: Track requests to manage rate limits
5. Handle partial failures: The analyzer returns partial results on errors

Troubleshooting

API Key Not Found

AI Defense API key required. Set AI_DEFENSE_API_KEY environment variable.

Solution: Export the environment variable or pass --aidefense-api-key flag.

Rate Limited

AI Defense API rate limited, retrying in 2s...

Solution: The analyzer automatically retries. For high-volume scanning, contact Cisco for rate limit increases.

Authentication Failed

Invalid AI Defense API key

Solution: Verify your API key is correct and active.

httpx Not Installed

httpx is required for AI Defense analyzer. Install with: pip install httpx

Solution: Install the required dependency:

pip install httpx

References

• Cisco AI Defense
• AI Defense API Documentation