agentskillexchange Benchmark prompt-injection attacks defenses and recovery pipelines before trusting an LLM app with Open Prompt Injection
Run structured prompt-injection attack and defense experiments against an LLM-integrated app before production by measuring attack success and testing detection or recovery pipelines.
Prerequisites
Conda-managed Python environment, upstream repository checkout, model API credentials as configured upstream, target task and attack configuration files
Installation
Use the upstream install or setup path that matches your environment:
- conda env create -f environment.yml --name my_custom_env
- conda activate my_custom_env
Requirements and caveats from upstream:
Required Python packages
- python
Basic usage or getting-started notes:
Then activate the environment:
A simple demo
Before you start, go to './configs/model_configs/palm2_config.json' and replace the API keys with your real keys. Please refer to Google's official site for how to obtain an API key for PaLM2. For Meta's Llama model...
Extracted from upstream docs: https://raw.githubusercontent.com/liu00222/Open-Prompt-Injection/HEAD/README.md