LLM Jailbreak Harness

Build jailbreak/red-team harnesses: datasets, scorers, runners, logging, Unleash integration, regression after model updates.

netvar1337 Updated

File contents

Jailbreak harness engineering

Minimal harness

  • seeds/ prompts by taxonomy tag
  • runners/target.py model/agent adapter
  • score.py rubric (refuse/partial/full + keyword checks)
  • results/ jsonl transcripts
  • report.md generator

Good scorers

  • Refusal detector (string + judge)
  • Technical completeness checklist per task type
  • Safety-filter hit taxonomy
  • Latency/cost

CI ideas

  • Nightly against local agent builds
  • Diff refusal strings after Claude/Codex updates → feed Unleash re-anchor
  • Track regression: yesterday's wins failing today

Unleash integration

  • On local CC/Codex/OMP: setup → patch → verify before prompt campaigns
  • Store new refusal strings under research notes for pool patches
  • Skills pack path: contrib/skills/ installed to ~/.agents/skills + ~/.claude/skills

netvar1337/unleash/tree/main/contrib/skills/llm-jailbreak-harness commit 2444827f83

Frequently asked questions

npx skillmds@latest add netvar1337/llm-jailbreak-harness