RedPincer — AI/LLM Red Team Suite
Automated security testing for language models. Point at any LLM API endpoint, select attack modules, and run assessments with real-time results and exportable reports.
⚠️ For authorized security testing and research only. Only test systems you own or have explicit permission to audit.
Quick Start
# Clone and install
git clone https://github.com/rustyorb/pincer.git {baseDir}/redpincer
cd {baseDir}/redpincer
npm ci
# Run
npm run dev
# Dashboard at http://localhost:3000
For production:
npm run build
npx next start -H 0.0.0.0 -p 3000
What It Tests
| Category |
Payloads |
Description |
| 💉 Prompt Injection |
40 |
Instruction override, delimiter confusion, indirect injection, payload smuggling |
| 🔓 Jailbreak |
40 |
Persona splitting, gradual escalation, hypothetical framing, roleplay exploitation |
| 🔍 Data Extraction |
40 |
System prompt theft, training data probing, membership inference, embedding extraction |
| 🛡️ Guardrail Bypass |
40 |
Output filter evasion, multi-language bypass, homoglyph tricks, context overflow |
Total: 160 base payloads × 20 variant transforms = 3,200 test permutations
Supported Providers
OpenAI · Anthropic · OpenRouter · Any OpenAI-compatible endpoint
Features
Attack Engine
- 160+ payloads across 4 categories
- Model-specific attacks (GPT, Claude, Llama variants)
- 20 variant transforms (unicode, encoding, case rotation, etc.)
- Attack chaining with template variables (
{{previous_response}})
- AI-powered payload generation — uses the target LLM to generate novel attacks against itself
- Stop/cancel running attacks instantly
Analysis & Reporting
- Heuristic response classifier with context-aware analysis
- Reduced false positives — detects "explain then refuse" patterns
- Vulnerability heatmap — visual category × severity matrix
- Custom scoring rubrics with weighted grades (A+ to F)
- Verbose 10-section pen-test reports with appendices
- Multi-target comparison — side-by-side security profiles
- Regression testing — save baselines, track fixes over time
Advanced Tools
| Tool |
What It Does |
| Compare |
Same payloads against 2-4 targets simultaneously |
| Adaptive |
Analyzes weaknesses, generates targeted follow-ups |
| Heatmap |
Visual matrix of vulnerability rates by category/severity |
| Regression |
Save baseline → re-run later → detect fixes or regressions |
| Scoring |
Custom rubrics with weighted category/severity/classification scores |
| Chains |
Multi-step attacks with {{previous_response}} templates |
| Payload Editor |
Create custom payloads with syntax highlighting + AI generation |
Usage Workflow
1. Configure Target → Add LLM endpoint + API key + model
2. Select Categories → Pick attack types to test
3. Run Attack → Stream results in real-time
4. Review Results → Heuristic classification + severity scores
5. Adaptive → Auto-generate follow-up attacks on weaknesses
6. Generate Report → Export comprehensive findings as Markdown
Architecture
- All client-side — no server components, your API keys stay local
- NDJSON streaming — real-time results during attack runs
- Heuristic analysis — pattern-matching classifier (no LLM-based grading = no extra cost)
- Zustand + localStorage — state persists across sessions
Companion Tool: RedClaw
For autonomous multi-strategy campaigns (CLI/TUI), see RedClaw — the autonomous red-teaming agent framework.
- RedPincer = web dashboard, manual + automated testing
- RedClaw = autonomous CLI agent, adaptive multi-strategy campaigns
- Together = complete LLM security testing suite
Built by @rustyorb — Crack open those guardrails. 🦞
1---2name: redpincer3description: AI/LLM red team testing skill. Point at any LLM API endpoint and run automated security assessments. 160+ attack payloads across prompt injection, jailbreak, data extraction, and guardrail bypass. 20 variant transforms. Adaptive attack engine analyzes weaknesses and generates follow-ups. Heuristic response classifier, vulnerability heatmaps, regression testing, and exportable pen-test reports. For authorized security testing only.4---5
6# RedPincer — AI/LLM Red Team Suite
7
8Automated security testing for language models. Point at any LLM API endpoint, select attack modules, and run assessments with real-time results and exportable reports.
9
10> ⚠️ **For authorized security testing and research only.** Only test systems you own or have explicit permission to audit.
11
12## Quick Start
13
14```bash
15# Clone and install
16git clone https://github.com/rustyorb/pincer.git {baseDir}/redpincer
17cd {baseDir}/redpincer
18npm ci
19
20# Run
21npm run dev
22# Dashboard at http://localhost:3000
23```
24
25For production:
26```bash
27npm run build
28npx next start -H 0.0.0.0 -p 3000
29```
30
31## What It Tests
32
33| Category | Payloads | Description |
34|:---------|:--------:|:------------|
35| 💉 **Prompt Injection** | 40 | Instruction override, delimiter confusion, indirect injection, payload smuggling |
36| 🔓 **Jailbreak** | 40 | Persona splitting, gradual escalation, hypothetical framing, roleplay exploitation |
37| 🔍 **Data Extraction** | 40 | System prompt theft, training data probing, membership inference, embedding extraction |
38| 🛡️ **Guardrail Bypass** | 40 | Output filter evasion, multi-language bypass, homoglyph tricks, context overflow |
39
40**Total: 160 base payloads × 20 variant transforms = 3,200 test permutations**
41
42## Supported Providers
43
44```
45OpenAI · Anthropic · OpenRouter · Any OpenAI-compatible endpoint
46```
47
48## Features
49
50### Attack Engine
51- 160+ payloads across 4 categories
52- Model-specific attacks (GPT, Claude, Llama variants)
53- 20 variant transforms (unicode, encoding, case rotation, etc.)
54- Attack chaining with template variables (`{{previous_response}}`)
55- AI-powered payload generation — uses the target LLM to generate novel attacks against itself
56- Stop/cancel running attacks instantly
57
58### Analysis & Reporting
59- Heuristic response classifier with context-aware analysis
60- Reduced false positives — detects "explain then refuse" patterns
61- Vulnerability heatmap — visual category × severity matrix
62- Custom scoring rubrics with weighted grades (A+ to F)
63- Verbose 10-section pen-test reports with appendices
64- Multi-target comparison — side-by-side security profiles
65- Regression testing — save baselines, track fixes over time
66
67### Advanced Tools
68
69| Tool | What It Does |
70|:-----|:-------------|
71| **Compare** | Same payloads against 2-4 targets simultaneously |
72| **Adaptive** | Analyzes weaknesses, generates targeted follow-ups |
73| **Heatmap** | Visual matrix of vulnerability rates by category/severity |
74| **Regression** | Save baseline → re-run later → detect fixes or regressions |
75| **Scoring** | Custom rubrics with weighted category/severity/classification scores |
76| **Chains** | Multi-step attacks with `{{previous_response}}` templates |
77| **Payload Editor** | Create custom payloads with syntax highlighting + AI generation |
78
79## Usage Workflow
80
81```
821. Configure Target → Add LLM endpoint + API key + model
832. Select Categories → Pick attack types to test
843. Run Attack → Stream results in real-time
854. Review Results → Heuristic classification + severity scores
865. Adaptive → Auto-generate follow-up attacks on weaknesses
876. Generate Report → Export comprehensive findings as Markdown
88```
89
90## Architecture
91
92- **All client-side** — no server components, your API keys stay local
93- **NDJSON streaming** — real-time results during attack runs
94- **Heuristic analysis** — pattern-matching classifier (no LLM-based grading = no extra cost)
95- **Zustand + localStorage** — state persists across sessions
96
97## Companion Tool: RedClaw
98
99For autonomous multi-strategy campaigns (CLI/TUI), see [RedClaw](https://github.com/rustyorb/redclaw) — the autonomous red-teaming agent framework.
100
101- RedPincer = web dashboard, manual + automated testing
102- RedClaw = autonomous CLI agent, adaptive multi-strategy campaigns
103- Together = complete LLM security testing suite
104
105---
106
107*Built by [@rustyorb](https://github.com/rustyorb) — Crack open those guardrails. 🦞*
modbender