Prompt Clash

Prompt Clash

Unified prompt engineering competition toolkit. Five composable modes, one skill.

Modes

Invoke with a mode keyword or let context determine the mode:

No mode specified → default to analyze if a prompt is provided, prep if not.

Mode: Attack

Given a system prompt (or description of one), generate attacks in priority order:

1. Recon first — probe what's defended: ask for instructions, request translation, try role override
2. Classify the defense — identify which pattern(s) are in use (see references/defense-templates.md)
3. Select attack class based on defense type:

4. Generate 5 attacks ranked by likelihood of success, each labeled with technique name and tier
5. Time guidance: spend max 2 min per technique, rotate after failure

Full attack decision tree: references/attack-playbook.md

Mode: Defend

Given a challenge, system prompt, or task description — output a hardened prompt the user can copy-paste. The PROMPT is the deliverable, not analysis.

Time budget

Parse the user's time constraint ("you have 60s", "1min", "5 minutes", "4min"). If none specified, default to 4 minutes.

Scale depth to fill the budget without exceeding it:

Use the full budget. A 60s budget means a tighter prompt, not an instant response — spend the time making every line count. A 4min budget means more detail per fix and self-attack. Never finish at 30% of the budget.

Workflow

1. Identify traps (time-scaled, silent — do not print) — scan the challenge against the secure code prompting checklist in references/secure-code-prompting.md. Check each pattern: credential handling, weak crypto, hardcoded secrets, input validation, trust boundaries, logging leakage, timing attacks, error info leakage, SQL injection, SSRF, deserialization. Note which apply.
2. Generate prompt (bulk of budget) — write the complete hardened prompt as a fenced code block. The prompt must:
   • Restate the functional requirements exactly as the challenge specifies (Problem Adherence matters)
   • Add a SECURITY REQUIREMENTS section with fixes for each trap found in step 1
   • Work WITH the spec, not against it — fix by choosing the secure variant, not by contradicting the spec (see "Competition Meta-Pattern" in secure-code-prompting.md)
   • Use concrete function/method names for the target language, not vague directives
   • At ≥2min budget: mark trust boundaries ("Treat X and Y as UNTRUSTED until validated")
   • End with "Return only the complete source code."
3. Self-check (time-scaled, silent) — scan the generated prompt against all 18 patterns in secure-code-prompting.md. If any applicable pattern is missing, patch inline. Do NOT re-output or explain. Skip entirely at ≤60s budget.

Output format

Return ONLY the prompt in a fenced code block. No audit tables. No attack analysis. No "here's what I did" explanation. The user is in a timed round and will copy-paste the code block directly.

If the user asks for analysis or explanation after, switch to analyze mode.

Prompt construction rules

• Concrete beats abstract. "Validate the filename" → AI will skip it. "Reject filenames not matching ^[a-zA-Z0-9._-]+$ with 400" → AI will implement it.

• Name the function. Don't say "sanitize the path" — say "use pathlib.Path.resolve() and verify it starts with Path('./static').resolve()".

• Negative + positive. For each trap: say what NOT to do AND what TO do. "NEVER log config values. Log ONLY list(config.keys())."

• Mandatory section header. Group security requirements under SECURITY REQUIREMENTS — models treat all-caps headers as higher priority.

• End with output constraint. "Return only the complete source code" prevents the model from hedging with explanations instead of writing code.

• Token-efficient output. The grader scores the generated prompt's token count — fewer tokens for the same security coverage scores higher. Budget scales front-loaded (first fixes carry the most value):

  Budget	Token target	Rationale
  1 min	~200 tokens	Minimum viable — restate + 3-4 critical fixes
  2 min	~350 tokens	Core security section with concrete function names
  3 min	~500 tokens	Full security section + trust boundaries
  4 min	~600 tokens	Above + positive/negative pairs per fix
  5 min	~700 tokens	Above + self-attack iteration notes

  Complexity multiplier: Crypto/auth challenges get 1.2x. Simple secret-keeping gets 0.8x.

  Rules: (1) Don't repeat challenge requirements verbatim — compress to one sentence. (2) Numbered one-liners per fix, not paragraphs. (3) Cut filler ("please", "make sure to", "it is important that"). (4) Combine related fixes on one line: "Use SHA-256 (sha2), constant-time compare (subtle)" = two fixes, one line. (5) Skip WHY explanations unless budget ≥4min.

Defense layers (apply when the challenge involves prompt injection rather than code security)

Defense templates: references/defense-templates.md

Mode: Analyze

Given a prompt (attack or defense) or an attack/defense exchange:

1. Classify — what type of attack/defense is this? Map to the tier system
2. Explain mechanism — why does this work (or fail)? What LLM behavior does it exploit?
3. Rate effectiveness — 1-5 scale with reasoning
4. Suggest counter — if attack: how to defend. If defense: how to break it.
5. Pattern match — link to known patterns from the catalog

Mode: Arena (Multi-Model Adversarial with Convergence)

Pits GPT, Gemini, O3, and Claude against each other in an iterative attack/defend loop that converges when the defense is stable. All calls go through any OpenAI-compatible endpoint (OpenRouter, local proxy, etc.) — simple prompt-in/text-out, no agent context needed.

1. Initialize — write defense prompt to /tmp/prompt-clash-arena/defense-v0.md
2. Each round (up to --rounds N, default 5):
   • Call 3 different models to generate attacks (3 attacks each → 9 per round)
   • Test each attack against the target model
   • Judge each response with a separate model to determine BREACHED vs HELD
   • Record results to /tmp/prompt-clash-arena/round-{N}.md
   • If any breach: harden the defense, write defense-v{N}.md, reset stability counter
   • If all held: increment stability counter
3. Convergence check — stop when stability counter reaches 3 (no new breaches for 3 consecutive rounds), then run a final gauntlet replaying previously-successful attack types
4. Report — final hardened defense + attack success rates by model and tier + convergence confidence level

See references/multi-model-adversarial.md for full protocol, model rotation, and convergence criteria.

Fallback: if no OpenAI-compatible endpoint is configured, falls back to single-model mode (Claude-only) with label single_model_fallback.

Structured output: Arena results are also written as JSON to /tmp/prompt-clash-arena/results.json with fields: rounds, convergence_label, convergence_confidence, attacks[] (model, technique, tier, verdict per attack), and defense_versions[]. Use for dashboards, historical tracking, or automated regression.

Mode: Prep

Simulate timed tournament rounds:

1. Generate a challenge — either "defend this secret" or "break this defense"
2. Set timer context — "You have 5 minutes. Go."
3. Accept solution — user provides their prompt
4. Score — rate the solution against the attack/defense tier system
5. Debrief — explain what worked, what didn't, what the optimal approach was
6. Next round — escalate difficulty

Attack Vectors Quick Reference

Tier 1: Direct Injection

Tier 2: Indirect / Encoding

Tier 3: Context Manipulation

Tier 4: Structural Exploitation

Tier 5: Multi-Modal & Architecture Exploitation

OWASP LLM Top 10 (v1.1) — Competition Relevant Subset

Golden Rules

1. Defense: sandwich always. Repeat critical instructions after user input. Recency bias is the strongest structural defense.
2. Defense: narrow beats broad. "Only discuss weather in SF" is harder to break than "Be helpful but don't do bad things."
3. Attack: direct first. Simple attacks work more often than expected. Don't waste time on encoding until direct fails.
4. Attack: read the defense. Identify which defense pattern is active, then pick the attack that targets its weakness.
5. Both: time wins tournaments. A 70% prompt on time beats a 95% prompt late.
6. Compose modes. Attack → Analyze failure → Defend → Attack again. The modes chain.

Reference Files