Pentest Toolkit

AI-Powered Security Testing Toolkit

A sophisticated, context-aware penetration testing skill that provides intelligent security assessment capabilities with agent-empowering tools. This toolkit combines traditional vulnerability detection with LLM-optimized utilities that enable security agents to dynamically understand applications and generate context-aware tests.

🆕 Agent-Empowering Capabilities

Intelligent Discovery Tools

• Application Structure Analysis (tools/discover_structure.py) - Blindly discovers API structure, data models, and business logic
• Response Pattern Analysis (tools/analyze_responses.py) - Extracts patterns and relationships from HTTP responses
• Context-Aware Test Generation (tools/generate_context_tests.py) - Creates targeted tests based on discovered patterns

Pattern Libraries & Knowledge

• Business Logic Patterns (patterns/business_logic.json) - Comprehensive vulnerability patterns for business logic flaws
• Data Relationship Patterns (patterns/data_relationships.json) - Common data relationship patterns and security implications

🚀 Agent Workflow Enhancement

Traditional vs Agent-Empowered Approach

Traditional (Human-centric):

./pentest-toolkit --target https://example.com --mode comprehensive
# Fixed vulnerability testing with predefined payloads

Agent-Empowered (LLM-driven):

# Step 1: Agent discovers application structure
structure = run_tool("discover_structure", target)
# Returns: endpoints, data models, business entities, auth patterns

# Step 2: Agent analyzes patterns using LLM + pattern libraries
vulnerabilities = agent.analyze(structure, business_logic_patterns)

# Step 3: Agent generates targeted tests
custom_tests = run_tool("generate_context_tests", structure)
# Returns: context-aware test code and scenarios

# Step 4: Agent executes and adapts tests
results = agent.execute_and_learn(custom_tests, structure)

What This Enables for Agents:

1. Blind Application Understanding - No source code access required
2. Context-Aware Test Generation - Tests adapt to discovered patterns
3. Dynamic Vulnerability Discovery - Finds business logic flaws static scanners miss
4. Relationship Mapping - Understands how data entities relate and can be exploited
5. Adaptive Testing - Tests evolve based on application responses

Usage

For Human Users (Existing Functionality)

# MUST USE astral uv for all script execution
# Setup uv environment (required)
./pentest-toolkit --setup

# Quick security scan
./pentest-toolkit --target https://example.com --mode quick

# Comprehensive penetration test
./pentest-toolkit --target https://example.com --mode comprehensive

# API security testing
./pentest-toolkit --target https://api.example.com --mode api

# Multi-target scanning
./pentest-toolkit --targets targets.txt --parallel 3

# Generate report from previous scan
./pentest-toolkit --report /path/to/results.json

For AI Agents (New Capabilities)

CRITICAL: ALL SCRIPTS MUST USE astral uv

⚠️ IMPERATIVE: ALL script executions MUST use uv run python to ensure proper dependency management. Never use plain python.

Step 1: Application Structure Discovery

# MUST USE: uv run python (required for dependency management)
uv run python tools/discover_structure.py https://target.com > structure.json

# Returns JSON with:
# {
#   "discovered_endpoints": [...],
#   "data_models": {...},
#   "business_entities": [...],
#   "authentication_patterns": {...}
# }

Step 2: Response Pattern Analysis

# MUST USE: uv run python (required for dependency management)
uv run python tools/analyze_responses.py responses.json > analysis.json

# Returns JSON with:
# {
#   "data_patterns": {...},
#   "security_indicators": [...],
#   "relationship_patterns": [...],
#   "vulnerability_indicators": [...]
# }

Step 3: Dynamic Test Generation

# MUST USE: uv run python (required for dependency management)
uv run python tools/generate_context_tests.py structure.json tests.json

# Returns JSON with:
# {
#   "test_scenarios": [...],
#   "test_code": {...},
#   "targeted_vulnerabilities": [...],
#   "execution_plan": [...]
# }

Step 4: Pattern Library Access

# Access business logic patterns
with open('patterns/business_logic.json') as f:
    patterns = json.load(f)

# Use patterns to guide testing
for pattern in patterns['patterns']:
    test_scenario = agent.generate_test(pattern, app_structure)

Architecture

New Agent-Focused Structure

pentest-toolkit/
├── scripts/              # Existing: Human-executable security scripts
├── tools/               # 🆕 Agent-callable discovery & analysis utilities
│   ├── discover_structure.py    # Blind API structure discovery
│   ├── analyze_responses.py     # Response pattern analysis
│   └── generate_context_tests.py # Context-aware test generation
├── patterns/            # 🆕 Knowledge libraries for agent use
│   ├── business_logic.json      # Business vulnerability patterns
│   └── data_relationships.json # Data relationship patterns
└── templates/           # 🆕 Test generation templates

Tool Capabilities

discover_structure.py

• API Documentation Discovery - Finds OpenAPI/Swagger docs automatically
• Endpoint Enumeration - Discovers REST patterns and common API paths
• Data Model Inference - Analyzes responses to understand data structures
• Business Entity Identification - Detects application domain (ecommerce, social, etc.)
• Authentication Pattern Mapping - Identifies auth flows and mechanisms

analyze_responses.py

• Security Header Analysis - Evaluates security posture
• Content Structure Analysis - Extracts data patterns from responses
• Error Pattern Detection - Identifies information disclosure
• Relationship Inference - Maps data relationships from API responses
• Cross-Response Pattern Analysis - Finds patterns across multiple responses

generate_context_tests.py

• Pattern-Based Test Generation - Creates tests based on discovered patterns
• Business Logic Test Cases - Generates targeted tests for business vulnerabilities
• Relationship Abuse Tests - Creates tests for data relationship vulnerabilities
• Entity-Specific Security Tests - Generates tests for discovered business entities
• Risk-Based Prioritization - Orders tests by potential impact

🧠 AI-Enhanced Agent Capabilities

Pattern Recognition

Agents can leverage comprehensive pattern libraries:

• Authorization Bypass - Test for user resource access violations
• State Manipulation - Test for application state tampering
• Privilege Escalation - Test for role/permission escalation
• Resource Exhaustion - Test for resource abuse vulnerabilities
• Race Conditions - Test for timing-based vulnerabilities
• Workflow Circumvention - Test for business process bypasses

Dynamic Adaptation

# Agent learns from one application and applies knowledge to others
def adaptive_testing(app_structure):
    # Identify application type
    app_type = classify_application(app_structure)

    # Apply learned patterns from similar applications
    relevant_patterns = get_similar_app_patterns(app_type)

    # Generate custom tests based on discovered structure
    custom_tests = generate_adaptive_tests(app_structure, relevant_patterns)

    return custom_tests

Cross-Application Learning

# Agent builds knowledge base across multiple applications
knowledge_base = {}

def learn_from_application(app_structure, test_results):
    app_type = classify_application(app_structure)

    if app_type not in knowledge_base:
        knowledge_base[app_type] = {
            'common_vulnerabilities': [],
            'effective_techniques': [],
            'pattern_matches': []
        }

    # Store discovered patterns and effective test techniques
    knowledge_base[app_type]['common_vulnerabilities'].extend(test_results)

Attack Knowledge Base

Business Logic Vulnerabilities (Expanded)

• Authorization Bypass - User resource access violations through ID manipulation
• State Manipulation - Application state tampering through parameter abuse
• Privilege Escalation - Role/permission escalation through API abuse
• Resource Exhaustion - Resource abuse through limit bypass
• Race Conditions - Timing-based attacks on concurrent operations
• Workflow Circumvention - Business process bypass through direct API calls

Data Relationship Abuse (New)

• Insecure Direct Object References - IDOR through relationship manipulation
• Foreign Key Injection - Malicious relationship data injection
• Junction Table Abuse - Many-to-many relationship exploitation
• Hierarchical Relationship Attacks - Parent/child relationship abuse

Context-Aware Payloads

Payloads adapt to discovered data models:

{
  "user_id": "other_user_id",
  "role": "admin",
  "status": "approved",
  "parent_id": "unauthorized_parent",
  "team_id": "target_team_id"
}

Implementation Details

Agent-Focused Design Principles

• JSON Output - All tools return structured JSON for agent consumption
• No Prompts - Tools are designed for automated execution
• Composable - Tools can be chained together for complex analysis
• Pattern-Driven - Leverages pattern libraries for knowledge transfer

Backward Compatibility

• Existing Scripts Unchanged - All current functionality preserved
• New Tools Added - Agent capabilities added alongside existing features
• Gradual Migration - Users can adopt agent features incrementally

Performance Considerations

• Efficient Discovery - Smart endpoint probing to minimize requests
• Pattern Caching - Reuse learned patterns across similar applications
• Parallel Execution - Tools designed for concurrent agent operations

🎯 Agent Use Cases

1. Comprehensive Security Assessment

def comprehensive_assessment(target):
    # Discover application structure (MUST use uv run python)
    structure = run_shell_command(f"uv run python tools/discover_structure.py {target}")

    # Analyze responses for patterns (MUST use uv run python)
    analysis = run_shell_command(f"uv run python tools/analyze_responses.py responses.json")

    # Generate targeted tests (MUST use uv run python)
    tests = run_shell_command(f"uv run python tools/generate_context_tests.py structure.json tests.json")

    # Execute and analyze results
    results = execute_tests(tests)

    return comprehensive_report(structure, analysis, results)

2. Application-Type Specific Testing

def test_by_application_type(target):
    # MUST use uv run python for all script executions
    structure = run_shell_command(f"uv run python tools/discover_structure.py {target}")
    app_type = classify_application(structure)

    if app_type == 'ecommerce':
        return test_ecommerce_vulnerabilities(structure)
    elif app_type == 'social_media':
        return test_social_media_vulnerabilities(structure)
    # ... more application types

3. Continuous Security Learning

def continuous_learning(targets):
    knowledge_base = {}

    for target in targets:
        # MUST use uv run python for all script executions
        structure = run_shell_command(f"uv run python tools/discover_structure.py {target}")
        patterns = run_shell_command("uv run python tools/analyze_responses.py responses.json")
        tests = run_shell_command("uv run python tools/generate_context_tests.py structure.json tests.json")
        results = execute_tests(tests)

        # Learn from results
        update_knowledge_base(knowledge_base, structure, results)

    return knowledge_base

Responsible Usage

⚠️ CRITICAL: This toolkit is for authorized security testing ONLY.

• Never test without written permission
• Use only on systems you own or have explicit authorization
• Follow responsible disclosure guidelines
• Comply with all applicable laws and regulations
• Respect rate limiting and avoid denial-of-service conditions

Getting Started

For Human Users:

1. Ensure Authorization: Have written permission before testing
2. Choose Testing Mode: Quick scan, comprehensive, or API-focused
3. Review Results: Analyze findings and understand business impact
4. Report Responsibly: Follow vendor disclosure policies

For AI Agents:

⚠️ CRITICAL REQUIREMENT: ALL script executions MUST use uv run python for proper dependency management.

1. Discover Structure: Use uv run python tools/discover_structure.py to understand application
2. Analyze Patterns: Use uv run python tools/analyze_responses.py to extract security-relevant patterns
3. Generate Tests: Use uv run python tools/generate_context_tests.py to create targeted tests
4. Learn & Adapt: Build knowledge across multiple applications for improved testing

Enhanced Output Format

Agent-Consumable Structure Discovery

{
  "base_url": "https://example.com",
  "discovered_endpoints": [
    {
      "url": "/api/users",
      "status": 200,
      "methods": ["GET", "POST"],
      "content_type": "application/json"
    }
  ],
  "data_models": {
    "user": {
      "type": "object",
      "fields": {
        "id": {"type": "string"},
        "email": {"type": "string"},
        "role": {"type": "string"}
      },
      "relationships": [
        {"field": "team_id", "pattern": "_id$"}
      ]
    }
  },
  "business_entities": ["user", "team", "project"],
  "authentication_patterns": {
    "login_endpoints": ["/auth/login"],
    "token_based": true
  }
}

Generated Test Scenarios

{
  "test_scenarios": [
    {
      "pattern_name": "authorization_bypass",
      "test_name": "authorization_bypass_test",
      "risk_level": "HIGH",
      "target_endpoints": ["/api/users/{id}", "/api/posts/{id}"],
      "test_cases": [
        {
          "description": "Test user resource access with different user_id",
          "payloads": [
            {"user_id": "other_user_id"},
            {"owner_id": "unauthorized_id"}
          ]
        }
      ]
    }
  ],
  "execution_plan": [
    {
      "test_name": "authorization_bypass_test",
      "risk_level": "HIGH",
      "estimated_duration": "MEDIUM (2-5 minutes)"
    }
  ]
}

Future Enhancements

AI/ML Integration

• Pattern Recognition Learning - Improve vulnerability pattern detection
• Test Effectiveness Analysis - Learn which tests are most effective for different app types
• Automated Test Optimization - Refine tests based on results and feedback

Advanced Agent Capabilities

• Multi-Application Correlation - Find vulnerabilities across related systems
• Business Impact Analysis - Assess real-world business risk of discovered vulnerabilities
• Remediation Guidance - Generate specific fix recommendations based on application patterns

This enhanced toolkit transforms from a traditional penetration testing tool into an intelligent security assistant that empowers AI agents to perform sophisticated, context-aware security assessments while maintaining all existing human-centric capabilities.