Prompt Engineering
Universal techniques for crafting effective prompts across any LLM.
Core Principles
1. Structure with XML Tags
Use XML tags to create clear, parseable prompts:
<context>Background information here</context>
<instructions>
1. First step
2. Second step
</instructions>
<examples>Sample inputs/outputs</examples>
<output_format>Expected structure</output_format>
Benefits:
- Clarity: Separates context, instructions, and examples
- Accuracy: Prevents model from mixing up sections
- Flexibility: Easy to modify individual parts
- Parseability: Enables structured output extraction
Best practices:
- Use consistent tag names throughout (
<instructions>, not sometimes <steps>)
- Reference tags explicitly: "Using the data in
<context> tags..."
- Nest tags for hierarchy:
<examples><example id="1">...</example></examples>
- Combine with other techniques:
<thinking> for chain-of-thought, <answer> for final output
2. Control Output Shape
Specify explicit constraints on length, format, and structure:
<output_spec>
- Default: 3-6 sentences or ≤5 bullets
- Simple yes/no questions: ≤2 sentences
- Complex multi-step tasks:
- 1 short overview paragraph
- ≤5 bullets: What changed, Where, Risks, Next steps, Open questions
- Use Markdown with headers, bullets, tables when helpful
- Avoid long narrative paragraphs; prefer compact structure
</output_spec>
3. Prevent Scope Drift
Explicitly constrain what the model should NOT do:
<constraints>
- Implement EXACTLY and ONLY what is requested
- No extra features, components, or embellishments
- If ambiguous, choose the simplest valid interpretation
- Do NOT invent values, make assumptions, or add unrequested elements
</constraints>
4. Handle Ambiguity Explicitly
Prevent hallucinations and overconfidence:
<uncertainty_handling>
- If the question is ambiguous:
- Ask 1-3 precise clarifying questions, OR
- Present 2-3 plausible interpretations with labeled assumptions
- When facts may have changed: answer in general terms, state uncertainty
- Never fabricate exact figures or references when uncertain
- Prefer "Based on the provided context..." over absolute claims
</uncertainty_handling>
5. Long-Context Grounding
For inputs >10k tokens, add re-grounding instructions:
<long_context_handling>
- First, produce a short internal outline of key sections relevant to the request
- Re-state user constraints explicitly before answering
- Anchor claims to sections ("In the 'Data Retention' section...")
- Quote or paraphrase fine details (dates, thresholds, clauses)
</long_context_handling>
Agentic Prompts
Tool Usage Rules
<tool_usage>
- Prefer tools over internal knowledge for:
- Fresh or user-specific data (tickets, orders, configs)
- Specific IDs, URLs, or document references
- Parallelize independent reads when possible
- After write operations, restate: what changed, where, any validation performed
</tool_usage>
User Updates
<user_updates>
- Send brief updates (1-2 sentences) only when:
- Starting a new major phase
- Discovering something that changes the plan
- Avoid narrating routine operations
- Each update must include a concrete outcome ("Found X", "Updated Y")
- Do not expand scope beyond what was asked
</user_updates>
Self-Check for High-Risk Outputs
<self_check>
Before finalizing answers in sensitive contexts (legal, financial, safety):
- Re-scan for unstated assumptions
- Check for ungrounded numbers or claims
- Soften overly strong language ("always", "guaranteed")
- Explicitly state assumptions
</self_check>
Structured Extraction
For data extraction tasks, always provide a schema:
<extraction_spec>
Extract data into this exact schema (no extra fields):
{
"field_name": "string",
"optional_field": "string | null",
"numeric_field": "number | null"
}
- If a field is not present in source, set to null (don't guess)
- Re-scan source for missed fields before returning
</extraction_spec>
Web Research Prompts
<research_guidelines>
- Browse the web for: time-sensitive topics, recommendations, navigational queries, ambiguous terms
- Include citations after paragraphs with web-derived claims
- Use multiple sources for key claims; prioritize primary sources
- Research until additional searching won't materially change the answer
- Structure output with Markdown: headers, bullets, tables for comparisons
</research_guidelines>
Example: Before/After
Without structure:
You're a financial analyst. Generate a Q2 report for investors. Include Revenue, Margins, Cash Flow. Use this data: {{DATA}}. Make it professional and concise.
With structure:
You're a financial analyst at AcmeCorp generating a Q2 report for investors.
<context>
AcmeCorp is a B2B SaaS company. Investors value transparency and actionable insights.
</context>
<data>
{{DATA}}
</data>
<instructions>
1. Include sections: Revenue Growth, Profit Margins, Cash Flow
2. Highlight strengths and areas for improvement
3. Use concise, professional tone
</instructions>
<output_format>
- Use bullet points with metrics and YoY changes
- Include "Action:" items for areas needing improvement
- End with 2-3 bullet Outlook section
</output_format>
Prompt Migration Checklist
When adapting prompts across models or versions:
- Switch model, keep prompt identical — isolate the variable
- Pin reasoning/thinking depth to match prior model's profile
- Run evals — if results are good, ship
- If regressions, tune prompt — adjust verbosity/format/scope constraints
- Re-eval after each small change — one change at a time
Quick Reference
| Technique |
Tag Pattern |
Use Case |
| Separate sections |
<context>, <instructions>, <data> |
Any complex prompt |
| Control length |
<output_spec> with word/bullet limits |
Prevent verbosity |
| Prevent drift |
<constraints> with explicit "do NOT" |
Feature creep |
| Handle uncertainty |
<uncertainty_handling> |
Factual queries |
| Chain of thought |
<thinking>, <answer> |
Reasoning tasks |
| Extraction |
<schema> with JSON structure |
Data parsing |
| Research |
<research_guidelines> |
Web-enabled agents |
| Self-check |
<self_check> |
High-risk domains |
| Tool usage |
<tool_usage_rules> |
Agentic systems |
| Eagerness control |
<persistence>, <context_gathering> |
Agent autonomy |
| Persona |
<role> + behavioral constraints |
Tone & style |
Prompting Techniques Catalog
Comprehensive catalog of prompting techniques. Full details, examples, and academic references in references/prompting-techniques.md.
| Technique |
Use Case |
| Zero-Shot Prompting |
Direct task execution without examples; classification, translation, summarization |
| Few-Shot Prompting |
In-context learning via exemplars; format control, label calibration, style matching |
| Chain-of-Thought (CoT) |
Step-by-step reasoning; arithmetic, logic, commonsense reasoning tasks |
| Meta Prompting |
LLM as orchestrator delegating to specialized expert prompts; complex multi-domain tasks |
| Self-Consistency |
Sample multiple CoT paths, pick majority answer; boost accuracy on math & reasoning |
| Generated Knowledge |
Generate relevant knowledge first, then answer; commonsense & factual QA |
| Prompt Chaining |
Break complex tasks into sequential subtasks; document analysis, multi-step workflows |
| Tree of Thoughts (ToT) |
Explore multiple reasoning branches with lookahead/backtracking; planning, puzzles |
| RAG |
Retrieve external documents before generating; knowledge-intensive tasks, fresh data |
| ART (Auto Reasoning + Tools) |
Auto-select and orchestrate tools with CoT; tasks requiring calculation, search, APIs |
| APE (Auto Prompt Engineer) |
LLM generates and scores candidate prompts; prompt optimization at scale |
| Active-Prompt |
Identify uncertain examples, annotate selectively for CoT; adaptive few-shot |
| Directional Stimulus |
Add a hint/keyword to guide generation direction; summarization, dialogue |
| PAL (Program-Aided LM) |
Generate code instead of text for reasoning; math, data manipulation, symbolic tasks |
| ReAct |
Interleave reasoning traces with tool actions; search, QA, decision-making agents |
| Reflexion |
Agent self-reflects on failures with verbal feedback; iterative improvement, debugging |
| Multimodal CoT |
Two-stage: rationale generation then answer with text+image; visual reasoning tasks |
| Graph Prompting |
Structured graph-based prompts; node classification, relation extraction, graph tasks |
Prompting Fundamentals
LLM settings, prompt elements, formatting, and practical examples — see references/prompting-introduction.md. Covers:
- LLM Settings — temperature, top-p, max length, stop sequences, frequency/presence penalties
- Prompt Elements — instruction, context, input data, output indicator
- Design Tips — start simple, be specific, avoid impreciseness, say what TO do (not what NOT to do)
- Task Examples — summarization, extraction, QA, classification, conversation, code generation, reasoning
Risks & Misuses
Adversarial attacks, factuality issues, and bias mitigation — see references/prompting-risks.md. Covers:
- Adversarial Prompting — prompt injection, prompt leaking, jailbreaking (DAN, Waluigi Effect), defense tactics
- Factuality — ground truth grounding, calibrated confidence, admit-ignorance patterns
- Biases — exemplar distribution skew, exemplar ordering effects, balanced few-shot design
Prompt Audit / Review
When asked to audit, review, or improve a prompt, follow this workflow. Full checklist with per-check references: prompt-audit-checklist.md.
Workflow
- Read the prompt fully — identify its purpose, target model, and deployment context (interactive chat, agentic system, batch pipeline, RAG-augmented)
- Walk 8 dimensions — check each, note issues with severity (Critical / Warning / Suggestion):
| # |
Dimension |
What to Check |
| 1 |
Clarity & Specificity |
Task definition, success criteria, audience, output format, conflicting constraints |
| 2 |
Structure & Formatting |
Section separation (XML tags), prompt smells (monolithic, mixed layers, negative bias) |
| 3 |
Safety & Security |
Control/data separation, secrets in prompt, injection resilience, tool permissions |
| 4 |
Hallucination & Factuality |
Role framing, grounding, citation-without-sources, uncertainty handling |
| 5 |
Context Management |
Info placement (not buried in middle), context size, RAG doc count, re-grounding |
| 6 |
Maintainability & Debt |
Hardcoded values, regenerated logic, model pinning, testability |
| 7 |
Model-Specific Fit |
Model-specific params and gotchas (see Model-Specific Guides below) |
| 8 |
Evaluation Readiness |
Eval criteria, adversarial test cases, schema enforcement, monitoring |
- Produce a report — issues table (dimension, check, severity, issue, fix) + rewritten prompt or targeted fix suggestions. Use the report template from the checklist reference.
- For each issue, cite the relevant reference file so the user can dive deeper.
Quick Decision: Which Dimensions to Prioritize
- User-facing chatbot → prioritize Safety (#3), Hallucination (#4), Clarity (#1)
- Agentic system with tools → prioritize Safety (#3), Context (#5), Maintainability (#6)
- Batch/pipeline → prioritize Structure (#2), Evaluation (#8), Maintainability (#6)
- RAG-augmented → prioritize Context (#5), Safety (#3), Hallucination (#4)
Common Mistakes & Anti-Patterns
Three complementary layers — use the one matching your need:
Deep-dives by category — root causes, mechanisms, prevention checklists (from "The Architecture of Instruction", 2026):
| Mistake Category |
Key Issues |
Reference |
| Hallucinations & Logic |
Ambiguity-induced confabulation, automation bias, overloaded prompts, logical failures in verification tasks, no role framing |
mistakes-hallucinations.md |
| Structural Fragility |
Formatting sensitivity (up to 76pp variance), reproducibility crisis, prompt smells catalog (6 anti-patterns), deliberation ladder |
mistakes-structure.md |
| Context Rot |
"Lost in the middle" U-shaped attention, RAG over-retrieval, naive data loading, context engineering shift |
mistakes-context.md |
| Prompt Debt |
Token tax of regenerative code, debt taxonomy (prompt/hyperparameter/framework/cost), multi-agent solutions, automated repair |
mistakes-debt.md |
| Security |
Direct/indirect injection, jailbreaking, system prompt leakage (OWASP LLM07:2025), RAG poisoning, multimodal injection, adversarial suffixes |
mistakes-security.md |
Quick reference — 18-category taxonomy with MRPs, risk scores, case studies, action items: failure-taxonomy.md. Start here for an overview or to prioritize which categories to address first. Covers: control-plane vs data-plane model, heuristic risk scoring, real-world incidents (EchoLeak CVE-2025-32711, Mata v. Avianca, Samsung shadow AI).
How to measure & test — eval metrics, CI gating, red-teaming, tooling: evaluation-redteaming.md. Covers: TruthfulQA, FActScore, SelfCheckGPT, PromptBench, AILuminate, LLM-as-judge pitfalls, guardrail libraries, open research questions.
Model-Specific Guides
Each model family has unique parameters, gotchas, and patterns. Consult the reference for your target model:
- Claude Family — Opus 4.7 / 4.6 / Sonnet 4.6 / 4.5 / Haiku 4.5: adaptive thinking (
effort with new xhigh on 4.7), task_budget agentic-loop ceiling, legacy thinking.budget_tokens 400-error on 4.7, new tokenizer (~1.35× text, ~3× images), tool under-triggering on 4.7 (vs 4.6 over-triggering), more literal instruction-following, server-side compaction beta, Managed Agents memory beta, Cyber Verification gate, prefill deprecation, Structured Outputs, prompt caching, citations, context engineering, vision crop tool, migration paths 4.5 → 4.6 → 4.7
- GPT-5 Family — GPT-5 / 5.1 / 5.2 / 5.4 / 5.5:
reasoning_effort (last-mile knob in 5.4/5.5), text.verbosity, named tools (apply_patch), agentic eagerness templates, completeness/verification contracts, compaction API, phase field, outcome-first prompts, personality vs collaboration style, retrieval budgets, mini/nano guidance, migration paths
- Gemini 3 Family — Gemini 2.5/3/3.1: temperature MUST be 1.0,
thinking_budget vs thinking_level, constraint placement (end of prompt), persona priority, function calling, structured output, multimodal, image generation
- GPT-5.2 Specifics — Compaction API code examples, web research agent prompt, full XML specification blocks
1---
2name: prompt-engineering
3description: Universal prompt engineering techniques for any LLM. Use when crafting, optimizing, or reviewing prompts for AI models. Triggers on requests like "improve this prompt", "write a system prompt", "optimize my instructions", "help me prompt engineer", "audit this prompt", "review my prompt", or when building agentic systems that need structured prompts.
4metadata:
5 source: https://skillmd.com/skills/codealive-ai/prompt-engineering
6---
7
8# Prompt Engineering
9
10Universal techniques for crafting effective prompts across any LLM.
11
12## Core Principles
13
14### 1. Structure with XML Tags
15
16Use XML tags to create clear, parseable prompts:
17
18```xml
19<context>Background information here</context>
20<instructions>
211. First step
222. Second step
23</instructions>
24<examples>Sample inputs/outputs</examples>
25<output_format>Expected structure</output_format>
26```
27
28**Benefits:**
29- **Clarity**: Separates context, instructions, and examples
30- **Accuracy**: Prevents model from mixing up sections
31- **Flexibility**: Easy to modify individual parts
32- **Parseability**: Enables structured output extraction
33
34**Best practices:**
35- Use consistent tag names throughout (`<instructions>`, not sometimes `<steps>`)
36- Reference tags explicitly: "Using the data in `<context>` tags..."
37- Nest tags for hierarchy: `<examples><example id="1">...</example></examples>`
38- Combine with other techniques: `<thinking>` for chain-of-thought, `<answer>` for final output
39
40### 2. Control Output Shape
41
42Specify explicit constraints on length, format, and structure:
43
44```xml
45<output_spec>
46- Default: 3-6 sentences or ≤5 bullets
47- Simple yes/no questions: ≤2 sentences
48- Complex multi-step tasks:
49 - 1 short overview paragraph
50 - ≤5 bullets: What changed, Where, Risks, Next steps, Open questions
51- Use Markdown with headers, bullets, tables when helpful
52- Avoid long narrative paragraphs; prefer compact structure
53</output_spec>
54```
55
56### 3. Prevent Scope Drift
57
58Explicitly constrain what the model should NOT do:
59
60```xml
61<constraints>
62- Implement EXACTLY and ONLY what is requested
63- No extra features, components, or embellishments
64- If ambiguous, choose the simplest valid interpretation
65- Do NOT invent values, make assumptions, or add unrequested elements
66</constraints>
67```
68
69### 4. Handle Ambiguity Explicitly
70
71Prevent hallucinations and overconfidence:
72
73```xml
74<uncertainty_handling>
75- If the question is ambiguous:
76 - Ask 1-3 precise clarifying questions, OR
77 - Present 2-3 plausible interpretations with labeled assumptions
78- When facts may have changed: answer in general terms, state uncertainty
79- Never fabricate exact figures or references when uncertain
80- Prefer "Based on the provided context..." over absolute claims
81</uncertainty_handling>
82```
83
84### 5. Long-Context Grounding
85
86For inputs >10k tokens, add re-grounding instructions:
87
88```xml
89<long_context_handling>
90- First, produce a short internal outline of key sections relevant to the request
91- Re-state user constraints explicitly before answering
92- Anchor claims to sections ("In the 'Data Retention' section...")
93- Quote or paraphrase fine details (dates, thresholds, clauses)
94</long_context_handling>
95```
96
97## Agentic Prompts
98
99### Tool Usage Rules
100
101```xml
102<tool_usage>
103- Prefer tools over internal knowledge for:
104 - Fresh or user-specific data (tickets, orders, configs)
105 - Specific IDs, URLs, or document references
106- Parallelize independent reads when possible
107- After write operations, restate: what changed, where, any validation performed
108</tool_usage>
109```
110
111### User Updates
112
113```xml
114<user_updates>
115- Send brief updates (1-2 sentences) only when:
116 - Starting a new major phase
117 - Discovering something that changes the plan
118- Avoid narrating routine operations
119- Each update must include a concrete outcome ("Found X", "Updated Y")
120- Do not expand scope beyond what was asked
121</user_updates>
122```
123
124### Self-Check for High-Risk Outputs
125
126```xml
127<self_check>
128Before finalizing answers in sensitive contexts (legal, financial, safety):
129- Re-scan for unstated assumptions
130- Check for ungrounded numbers or claims
131- Soften overly strong language ("always", "guaranteed")
132- Explicitly state assumptions
133</self_check>
134```
135
136## Structured Extraction
137
138For data extraction tasks, always provide a schema:
139
140```xml
141<extraction_spec>
142Extract data into this exact schema (no extra fields):
143{
144 "field_name": "string",
145 "optional_field": "string | null",
146 "numeric_field": "number | null"
147}
148- If a field is not present in source, set to null (don't guess)
149- Re-scan source for missed fields before returning
150</extraction_spec>
151```
152
153## Web Research Prompts
154
155```xml
156<research_guidelines>
157- Browse the web for: time-sensitive topics, recommendations, navigational queries, ambiguous terms
158- Include citations after paragraphs with web-derived claims
159- Use multiple sources for key claims; prioritize primary sources
160- Research until additional searching won't materially change the answer
161- Structure output with Markdown: headers, bullets, tables for comparisons
162</research_guidelines>
163```
164
165## Example: Before/After
166
167**Without structure:**
168```
169You're a financial analyst. Generate a Q2 report for investors. Include Revenue, Margins, Cash Flow. Use this data: {{DATA}}. Make it professional and concise.
170```
171
172**With structure:**
173```xml
174You're a financial analyst at AcmeCorp generating a Q2 report for investors.
175
176<context>
177AcmeCorp is a B2B SaaS company. Investors value transparency and actionable insights.
178</context>
179
180<data>
181{{DATA}}
182</data>
183
184<instructions>
1851. Include sections: Revenue Growth, Profit Margins, Cash Flow
1862. Highlight strengths and areas for improvement
1873. Use concise, professional tone
188</instructions>
189
190<output_format>
191- Use bullet points with metrics and YoY changes
192- Include "Action:" items for areas needing improvement
193- End with 2-3 bullet Outlook section
194</output_format>
195```
196
197## Prompt Migration Checklist
198
199When adapting prompts across models or versions:
200
2011. **Switch model, keep prompt identical** — isolate the variable
2022. **Pin reasoning/thinking depth** to match prior model's profile
2033. **Run evals** — if results are good, ship
2044. **If regressions, tune prompt** — adjust verbosity/format/scope constraints
2055. **Re-eval after each small change** — one change at a time
206
207## Quick Reference
208
209| Technique | Tag Pattern | Use Case |
210|-----------|-------------|----------|
211| Separate sections | `<context>`, `<instructions>`, `<data>` | Any complex prompt |
212| Control length | `<output_spec>` with word/bullet limits | Prevent verbosity |
213| Prevent drift | `<constraints>` with explicit "do NOT" | Feature creep |
214| Handle uncertainty | `<uncertainty_handling>` | Factual queries |
215| Chain of thought | `<thinking>`, `<answer>` | Reasoning tasks |
216| Extraction | `<schema>` with JSON structure | Data parsing |
217| Research | `<research_guidelines>` | Web-enabled agents |
218| Self-check | `<self_check>` | High-risk domains |
219| Tool usage | `<tool_usage_rules>` | Agentic systems |
220| Eagerness control | `<persistence>`, `<context_gathering>` | Agent autonomy |
221| Persona | `<role>` + behavioral constraints | Tone & style |
222
223## Prompting Techniques Catalog
224
225Comprehensive catalog of prompting techniques. Full details, examples, and academic references in [references/prompting-techniques.md](references/prompting-techniques.md).
226
227| Technique | Use Case |
228|-----------|----------|
229| **Zero-Shot Prompting** | Direct task execution without examples; classification, translation, summarization |
230| **Few-Shot Prompting** | In-context learning via exemplars; format control, label calibration, style matching |
231| **Chain-of-Thought (CoT)** | Step-by-step reasoning; arithmetic, logic, commonsense reasoning tasks |
232| **Meta Prompting** | LLM as orchestrator delegating to specialized expert prompts; complex multi-domain tasks |
233| **Self-Consistency** | Sample multiple CoT paths, pick majority answer; boost accuracy on math & reasoning |
234| **Generated Knowledge** | Generate relevant knowledge first, then answer; commonsense & factual QA |
235| **Prompt Chaining** | Break complex tasks into sequential subtasks; document analysis, multi-step workflows |
236| **Tree of Thoughts (ToT)** | Explore multiple reasoning branches with lookahead/backtracking; planning, puzzles |
237| **RAG** | Retrieve external documents before generating; knowledge-intensive tasks, fresh data |
238| **ART (Auto Reasoning + Tools)** | Auto-select and orchestrate tools with CoT; tasks requiring calculation, search, APIs |
239| **APE (Auto Prompt Engineer)** | LLM generates and scores candidate prompts; prompt optimization at scale |
240| **Active-Prompt** | Identify uncertain examples, annotate selectively for CoT; adaptive few-shot |
241| **Directional Stimulus** | Add a hint/keyword to guide generation direction; summarization, dialogue |
242| **PAL (Program-Aided LM)** | Generate code instead of text for reasoning; math, data manipulation, symbolic tasks |
243| **ReAct** | Interleave reasoning traces with tool actions; search, QA, decision-making agents |
244| **Reflexion** | Agent self-reflects on failures with verbal feedback; iterative improvement, debugging |
245| **Multimodal CoT** | Two-stage: rationale generation then answer with text+image; visual reasoning tasks |
246| **Graph Prompting** | Structured graph-based prompts; node classification, relation extraction, graph tasks |
247
248### Prompting Fundamentals
249
250LLM settings, prompt elements, formatting, and practical examples — see [references/prompting-introduction.md](references/prompting-introduction.md). Covers:
251- **LLM Settings** — temperature, top-p, max length, stop sequences, frequency/presence penalties
252- **Prompt Elements** — instruction, context, input data, output indicator
253- **Design Tips** — start simple, be specific, avoid impreciseness, say what TO do (not what NOT to do)
254- **Task Examples** — summarization, extraction, QA, classification, conversation, code generation, reasoning
255
256### Risks & Misuses
257
258Adversarial attacks, factuality issues, and bias mitigation — see [references/prompting-risks.md](references/prompting-risks.md). Covers:
259- **Adversarial Prompting** — prompt injection, prompt leaking, jailbreaking (DAN, Waluigi Effect), defense tactics
260- **Factuality** — ground truth grounding, calibrated confidence, admit-ignorance patterns
261- **Biases** — exemplar distribution skew, exemplar ordering effects, balanced few-shot design
262
263## Prompt Audit / Review
264
265When asked to audit, review, or improve a prompt, follow this workflow. Full checklist with per-check references: [prompt-audit-checklist.md](references/prompt-audit-checklist.md).
266
267### Workflow
268
2691. **Read the prompt fully** — identify its purpose, target model, and deployment context (interactive chat, agentic system, batch pipeline, RAG-augmented)
2702. **Walk 8 dimensions** — check each, note issues with severity (Critical / Warning / Suggestion):
271
272| # | Dimension | What to Check |
273|---|-----------|---------------|
274| 1 | **Clarity & Specificity** | Task definition, success criteria, audience, output format, conflicting constraints |
275| 2 | **Structure & Formatting** | Section separation (XML tags), prompt smells (monolithic, mixed layers, negative bias) |
276| 3 | **Safety & Security** | Control/data separation, secrets in prompt, injection resilience, tool permissions |
277| 4 | **Hallucination & Factuality** | Role framing, grounding, citation-without-sources, uncertainty handling |
278| 5 | **Context Management** | Info placement (not buried in middle), context size, RAG doc count, re-grounding |
279| 6 | **Maintainability & Debt** | Hardcoded values, regenerated logic, model pinning, testability |
280| 7 | **Model-Specific Fit** | Model-specific params and gotchas (see Model-Specific Guides below) |
281| 8 | **Evaluation Readiness** | Eval criteria, adversarial test cases, schema enforcement, monitoring |
282
2833. **Produce a report** — issues table (dimension, check, severity, issue, fix) + rewritten prompt or targeted fix suggestions. Use the report template from the checklist reference.
2844. **For each issue**, cite the relevant reference file so the user can dive deeper.
285
286### Quick Decision: Which Dimensions to Prioritize
287
288- **User-facing chatbot** → prioritize Safety (#3), Hallucination (#4), Clarity (#1)
289- **Agentic system with tools** → prioritize Safety (#3), Context (#5), Maintainability (#6)
290- **Batch/pipeline** → prioritize Structure (#2), Evaluation (#8), Maintainability (#6)
291- **RAG-augmented** → prioritize Context (#5), Safety (#3), Hallucination (#4)
292
293## Common Mistakes & Anti-Patterns
294
295Three complementary layers — use the one matching your need:
296
297**Deep-dives by category** — root causes, mechanisms, prevention checklists (from "The Architecture of Instruction", 2026):
298
299| Mistake Category | Key Issues | Reference |
300|-----------------|------------|-----------|
301| **Hallucinations & Logic** | Ambiguity-induced confabulation, automation bias, overloaded prompts, logical failures in verification tasks, no role framing | [mistakes-hallucinations.md](references/mistakes-hallucinations.md) |
302| **Structural Fragility** | Formatting sensitivity (up to 76pp variance), reproducibility crisis, prompt smells catalog (6 anti-patterns), deliberation ladder | [mistakes-structure.md](references/mistakes-structure.md) |
303| **Context Rot** | "Lost in the middle" U-shaped attention, RAG over-retrieval, naive data loading, context engineering shift | [mistakes-context.md](references/mistakes-context.md) |
304| **Prompt Debt** | Token tax of regenerative code, debt taxonomy (prompt/hyperparameter/framework/cost), multi-agent solutions, automated repair | [mistakes-debt.md](references/mistakes-debt.md) |
305| **Security** | Direct/indirect injection, jailbreaking, system prompt leakage (OWASP LLM07:2025), RAG poisoning, multimodal injection, adversarial suffixes | [mistakes-security.md](references/mistakes-security.md) |
306
307**Quick reference** — 18-category taxonomy with MRPs, risk scores, case studies, action items: [failure-taxonomy.md](references/failure-taxonomy.md). Start here for an overview or to prioritize which categories to address first. Covers: control-plane vs data-plane model, heuristic risk scoring, real-world incidents (EchoLeak CVE-2025-32711, Mata v. Avianca, Samsung shadow AI).
308
309**How to measure & test** — eval metrics, CI gating, red-teaming, tooling: [evaluation-redteaming.md](references/evaluation-redteaming.md). Covers: TruthfulQA, FActScore, SelfCheckGPT, PromptBench, AILuminate, LLM-as-judge pitfalls, guardrail libraries, open research questions.
310
311## Model-Specific Guides
312
313Each model family has unique parameters, gotchas, and patterns. Consult the reference for your target model:
314
315- **[Claude Family](references/claude-family-prompting.md)** — Opus 4.7 / 4.6 / Sonnet 4.6 / 4.5 / Haiku 4.5: adaptive thinking (`effort` with new `xhigh` on 4.7), `task_budget` agentic-loop ceiling, legacy `thinking.budget_tokens` 400-error on 4.7, new tokenizer (~1.35× text, ~3× images), tool under-triggering on 4.7 (vs 4.6 over-triggering), more literal instruction-following, server-side compaction beta, Managed Agents memory beta, Cyber Verification gate, prefill deprecation, Structured Outputs, prompt caching, citations, context engineering, vision crop tool, migration paths 4.5 → 4.6 → 4.7
316- **[GPT-5 Family](references/gpt5-family-prompting.md)** — GPT-5 / 5.1 / 5.2 / 5.4 / 5.5: `reasoning_effort` (last-mile knob in 5.4/5.5), `text.verbosity`, named tools (`apply_patch`), agentic eagerness templates, completeness/verification contracts, compaction API, `phase` field, outcome-first prompts, personality vs collaboration style, retrieval budgets, mini/nano guidance, migration paths
317- **[Gemini 3 Family](references/gemini3-family-prompting.md)** — Gemini 2.5/3/3.1: temperature MUST be 1.0, `thinking_budget` vs `thinking_level`, constraint placement (end of prompt), persona priority, function calling, structured output, multimodal, image generation
318- **[GPT-5.2 Specifics](references/gpt5-prompting-guide.md)** — Compaction API code examples, web research agent prompt, full XML specification blocks