AI Security Tooling
Scope
Use this skill when adding or organizing:
- LLM security tools (guardrails, detectors)
- Adversarial ML libraries
- AI vulnerability scanners
- Model safety tools
- Security benchmarks and frameworks
Tool Categories
LLM Security Tools
- Guardrails: NeMo Guardrails, LLM Guard, Rebuff
- Detectors: Vigil-LLM, Nova Framework, Garak
- Scanners: ModelScan, AI Security Analyzer
Adversarial ML Libraries
- Attack libraries: ART, CleverHans, Foolbox, TextAttack
- Defense libraries: SecML
- Fuzzing: OSS-Fuzz-Gen, Brainstorm
AI Red Teaming
- Microsoft: Counterfit, PyRIT
- Meta: PurpleLlama
- NVIDIA: Garak, NeMo Guardrails
Benchmarks
- Robustness: RobustBench
- Jailbreak: JailbreakBench
- Safety: Stanford AIR-Bench
- Hallucination: Vectara Leaderboard
Standards & Frameworks
- MITRE ATLAS: AI threat matrix
- NIST AI RMF: Risk management framework
- OWASP: LLM Top 10, GenAI Security Project
Categorization Rules
- LLM guardrails/detectors →
AI Security & Attacks → Model Security
- Prompt injection tools →
AI Security & Attacks → Prompt Injection
- Adversarial ML libraries →
AI Security & Attacks → Adversarial Attacks or AI Security Libraries
- AI RE/debugging tools →
AI Security Tools & Frameworks → AI Reverse Engineering
- AI vulnerability scanners →
AI Security Tools & Frameworks → AI Vulnerability Detection
- Benchmarks →
Benchmarks & Standards
- MCP security tools →
AI Pentesting & Red Teaming → AI Security MCP Tools
Quality Bar
- Prefer canonical repos
- Avoid forks unless they add meaningful features
- Add short descriptions
- Never duplicate an existing URL
- Tool must be AI/ML-focused
Key Vendor Tools
| Vendor |
Tools |
| Microsoft |
Counterfit, PyRIT |
| Meta |
PurpleLlama (Llama Guard, Prompt Guard, Code Shield) |
| NVIDIA |
Garak, NeMo Guardrails |
| IBM |
Adversarial Robustness Toolbox (ART) |
| Google |
OSS-Fuzz-Gen |
| ProtectAI |
Rebuff, LLM Guard, ModelScan |
Notes
Keep additions:
- AI/ML security focused
- Non-duplicated URLs
- Minimal structural changes
Data Source
For detailed and up-to-date resources, fetch the complete list from:
https://raw.githubusercontent.com/gmh5225/awesome-ai-security/refs/heads/main/README.md
Use this URL to get the latest curated links when you need specific tools, papers, or resources not covered in this skill.
1---2name: ai-security-tooling3description: Guide for AI security tooling (detectors, analyzers, guardrails, benchmarks) and consistent placement in README.md.4---56# AI Security Tooling78## Scope910Use this skill when adding or organizing:1112- LLM security tools (guardrails, detectors)13- Adversarial ML libraries14- AI vulnerability scanners15- Model safety tools16- Security benchmarks and frameworks1718## Tool Categories1920### LLM Security Tools21- **Guardrails**: NeMo Guardrails, LLM Guard, Rebuff22- **Detectors**: Vigil-LLM, Nova Framework, Garak23- **Scanners**: ModelScan, AI Security Analyzer2425### Adversarial ML Libraries26- **Attack libraries**: ART, CleverHans, Foolbox, TextAttack27- **Defense libraries**: SecML28- **Fuzzing**: OSS-Fuzz-Gen, Brainstorm2930### AI Red Teaming31- **Microsoft**: Counterfit, PyRIT32- **Meta**: PurpleLlama33- **NVIDIA**: Garak, NeMo Guardrails3435### Benchmarks36- **Robustness**: RobustBench37- **Jailbreak**: JailbreakBench38- **Safety**: Stanford AIR-Bench39- **Hallucination**: Vectara Leaderboard4041### Standards & Frameworks42- **MITRE ATLAS**: AI threat matrix43- **NIST AI RMF**: Risk management framework44- **OWASP**: LLM Top 10, GenAI Security Project4546## Categorization Rules4748- **LLM guardrails/detectors** → `AI Security & Attacks → Model Security`49- **Prompt injection tools** → `AI Security & Attacks → Prompt Injection`50- **Adversarial ML libraries** → `AI Security & Attacks → Adversarial Attacks` or `AI Security Libraries`51- **AI RE/debugging tools** → `AI Security Tools & Frameworks → AI Reverse Engineering`52- **AI vulnerability scanners** → `AI Security Tools & Frameworks → AI Vulnerability Detection`53- **Benchmarks** → `Benchmarks & Standards`54- **MCP security tools** → `AI Pentesting & Red Teaming → AI Security MCP Tools`5556## Quality Bar5758- Prefer canonical repos59- Avoid forks unless they add meaningful features60- Add short descriptions61- Never duplicate an existing URL62- Tool must be AI/ML-focused6364## Key Vendor Tools6566| Vendor | Tools |67|--------|-------|68| Microsoft | Counterfit, PyRIT |69| Meta | PurpleLlama (Llama Guard, Prompt Guard, Code Shield) |70| NVIDIA | Garak, NeMo Guardrails |71| IBM | Adversarial Robustness Toolbox (ART) |72| Google | OSS-Fuzz-Gen |73| ProtectAI | Rebuff, LLM Guard, ModelScan |7475## Notes7677Keep additions:7879- AI/ML security focused80- Non-duplicated URLs81- Minimal structural changes8283## Data Source8485For detailed and up-to-date resources, fetch the complete list from:8687```88https://raw.githubusercontent.com/gmh5225/awesome-ai-security/refs/heads/main/README.md89```9091Use this URL to get the latest curated links when you need specific tools, papers, or resources not covered in this skill.
gmh5225