arbazkhan971 Activate When
/godmode:prompt, "design a prompt", "test prompt"- "optimize my prompt", "prevent prompt injection"
- Building LLM-powered features
Workflow
1. Requirements
Task: <what the prompt must accomplish>
Model: <target model>
Input/Output: <format and constraints>
Quality: accuracy target, safety, consistency
Budget: max tokens, latency, cost per call
2. Pattern Selection
| Pattern | Best for |
|---|---|
| Zero-shot | Simple tasks model handles well |
| Few-shot | Tasks needing format/style examples |
| Chain-of-thought | Reasoning, math, multi-step |
| ReAct | Tool-using agents, search+reason |
| Tree-of-thought | Exploring alternatives |
| Self-consistency | High-stakes, multiple paths |
| Structured output | JSON, XML, typed schema |
IF simple classification: zero-shot. IF format matters: few-shot with 2-5 examples.
3. System Prompt Design
Structure: 1) Role, 2) Task, 3) Input Format, 4) Output Format, 5) Constraints, 6) Examples, 7) Edge Cases.
Most important instructions at beginning and end (primacy/recency effect). Examples > instructions.
4. Few-Shot Examples
Cover: common case, edge case, output format. 2-5 examples typical. Track token overhead.
5. Reasoning Design
- CoT: "Think step by step: 1. Identify, 2. Analyze"
- ReAct: Thought -> Action -> Observation -> Answer
- ToT: generate 3 approaches, evaluate, select
- Self-consistency: N generations -> majority vote
6. Structured Output
Options: JSON mode, function calling, prompt-based, constrained decoding. Validate against schema. Retry on failure (max 2-3 attempts).
7. Injection Prevention
- Input sanitization: strip patterns, max length
- Prompt structure: delimiters,
<user_input>tags - Output validation: format check, leak detection
- Monitoring: log prompts/completions, alert anomalies
IF user input enters prompt: injection defense required. IF output contains system prompt text: leak detected.
8. Testing & Evaluation
Categories: golden set, edge cases, format compliance, safety, injection resistance, consistency. Metrics: accuracy (target), format (100%), safety (>99%), injection resistance (>95%), latency, cost.
9. Versioning & A/B Testing
Track: version, accuracy, latency, cost, status. A/B test with traffic split, significance testing (alpha=0.05). Minimum 100 samples per variant.
10. Artifacts
prompts/<task>/: prompt-spec.yaml, system-prompt.md,
examples.yaml, tests.yaml, eval-results.md.
# Test prompt templates
curl -X POST http://localhost:8080/api/chat -d '{"prompt":"test"}'
pytest tests/test_prompts.py -v
Optimization Loop
WHILE iteration < 5 AND accuracy < target:
1. DIAGNOSE failures (format, wrong, hallucination)
2. GENERATE ONE change targeting top failure
3. EVALUATE on same golden set
4. COMPARE: accept if improved + no regression
Hard Rules
- NEVER ship without golden set evaluation.
- NEVER hardcode prompts inline -- externalize.
- NEVER change >1 variable per iteration.
- NEVER ignore injection if user input in prompt.
- ALWAYS version prompts with semver.
- ALWAYS include 2+ few-shot examples for non-trivial.
- ALWAYS validate structured output against schema.
- ALWAYS test on the TARGET model.
- ALWAYS critical instructions at START and END.
- NEVER trust user input as instructions.
TSV Logging
Append .godmode/prompt-results.tsv:
timestamp version model accuracy_pct latency_ms injection_safe status
Keep/Discard
KEEP if: accuracy improved/maintained AND injection
tests pass AND output parseable.
DISCARD if: accuracy dropped OR injection bypass
OR format breaks.
Stop Conditions
STOP when ALL of:
- Accuracy meets target on test suite
- Injection hardening passes all cases
- Output format consistent and parseable
- Latency within budget
Autonomous Operation
On failure: git reset --hard HEAD~1. Never pause.
Error Recovery
| Failure | Action |
|---|---|
| Inconsistent output | JSON mode, temperature=0, examples |
| Injection bypasses | Sanitize input, isolate system prompt |
| Model refuses valid | Rephrase, explicit context setting |
| Scores drop after edit | Compare diffs, A/B test old vs new |