build-with-dhiraj Prompt Engineering Reference
Based on the dair-ai Prompt Engineering Guide.
When to Apply
- Writing or refining system prompts for Claude features
- Designing few-shot examples or CoT demonstrations
- Building classification prompts with locked SME rubrics
- Structuring RAG retrieval prompts
- Debugging LLM outputs that miss intent
- Instrumenting prompts for eval before shipping
1. Prompt Anatomy
Every effective prompt has four addressable slots:
| Slot | Purpose | Required? |
|---|---|---|
| Instruction | What the model must do | Always |
| Context | Background the model needs | Usually |
| Input | The actual data to process | Usually |
| Output format | Shape of the expected response | Always |
Omitting output format is the single most common source of inconsistent LLM behaviour.
[SYSTEM / INSTRUCTION]
You are a classification assistant. Given a customer message, return a JSON object
with keys `category` (string) and `confidence` (0–1 float). No prose.
[CONTEXT]
Categories: billing, technical_support, general_inquiry, escalation.
[INPUT]
Customer message: "{{message}}"
[OUTPUT FORMAT]
{"category": "...", "confidence": 0.0}
2. Zero-shot → Few-shot → Chain-of-Thought Progression
Escalate only when the simpler technique fails.
| Technique | When to use | Trade-off |
|---|---|---|
| Zero-shot | Simple, well-defined tasks; model has strong priors | Fastest, cheapest |
| Few-shot | Task needs format adherence or domain nuance | +tokens per call |
| CoT (standard) | Multi-step reasoning; math; rubric scoring | Much larger context |
| Zero-shot CoT | CoT benefit without writing examples | Slightly less precise than standard CoT |
Rule of thumb: start zero-shot, add few-shot if format drifts, add CoT if reasoning is wrong.
3. Chain-of-Thought (CoT)
Standard CoT — include worked examples
Q: A rubric has 4 criteria each scored 1–5. Score: [3, 4, 2, 5]. Is the overall
score above 14?
A: Sum = 3+4+2+5 = 14. 14 is not above 14. Answer: No.
Q: Score: [4, 4, 3, 5]. Is the overall score above 14?
A: [model continues reasoning]
Zero-shot CoT — append the trigger phrase
Evaluate this response against the SME rubric. Think step by step before giving
your final score.
CoT for locked rubrics (Ask AI / Chakra pattern)
When scoring against a fixed SME rubric, force the model to cite the rubric criterion before assigning a score. This prevents hallucinated justifications.
For each criterion below, quote the relevant excerpt from the response, then
assign a score (1–5). Only after scoring all criteria give the aggregate.
4. Tool-Use / Function-Calling Prompts
Claude's tool-use depends on crisp tool descriptions. Treat each tool description as a mini prompt.
tools = [
{
"name": "search_knowledge_base",
"description": (
"Search the internal knowledge base for information relevant to a user question. "
"Use when the user asks about product features, pricing, or policies. "
"Do NOT use for general world knowledge — only internal docs."
),
"input_schema": {
"type": "object",
"properties": {
"query": {"type": "string", "description": "Concise search query, max 15 words"}
},
"required": ["query"]
}
}
]
Key rules:
- State what the tool does AND when NOT to use it
- Constrain input fields with descriptions and formats
- One tool per responsibility — avoid Swiss-army tool descriptions
5. Classification Prompt Template (SME Rubric)
[SYSTEM]
You are a strict evaluator using the rubric below. Return only valid JSON.
Do not infer intent beyond what is stated in the rubric.
RUBRIC (locked — do not modify):
{{rubric_text}}
[USER]
Evaluate the following response.
Response: """{{response_text}}"""
Reference answer: """{{reference_text}}"""
Output format:
{
"scores": {"criterion_1": <int 1-5>, "criterion_2": <int 1-5>, ...},
"aggregate": <float>,
"reasoning": "<one sentence per criterion>"
}
Failure modes to avoid:
- Leaking the rubric-modification instruction into the user turn
- Asking for scores and prose in the same turn without a separator
- Omitting the reference answer when the rubric is comparative
6. RAG Prompt Template
[SYSTEM]
Answer the user's question using ONLY the provided context.
If the context does not contain sufficient information, say "I don't have enough
information to answer that." Do not fabricate facts.
[CONTEXT]
{{retrieved_chunks}}
[USER QUESTION]
{{question}}
Checklist before shipping a RAG prompt:
- Grounding instruction is in the SYSTEM turn, not user turn
- Model is told what to say when context is insufficient
- Retrieved chunks are clearly delimited (e.g.,
---between chunks) - Output format is specified if structured output is needed
7. Common Failure Modes
| Failure | Symptom | Fix |
|---|---|---|
| Under-specification | Output length/format varies wildly | Add explicit output format slot |
| Conflicting instructions | Model alternates between behaviours | Audit system + user turns for contradictions; put the authoritative rule in SYSTEM |
| Role confusion | Model breaks character or ignores persona | Define role with negative examples: "You are X. Do not do Y." |
| Prompt injection | User input overrides system instructions | Delimit user content: User input: """{{input}}""" |
| Hallucinated tool calls | Model invents tool arguments | Add "required" fields + description constraints to input schema |
| Rubric drift | Scores shift across runs with same input | Pin temperature to 0; add CoT + cite-before-score pattern |
8. Prompt Versioning Discipline
Treat prompts as code. A prompt change without an eval is a blind deploy.
prompts/
classifier/
v1.0.0.md # baseline
v1.1.0.md # added CoT
v1.2.0.md # tightened output format
rag/
v1.0.0.md
Checklist before shipping a prompt change:
- New version file committed (never overwrite in place)
- Eval run on held-out set (same set across versions for comparability)
- Delta metrics recorded: accuracy, format compliance, latency, cost
- Rollback path tested (old version still in repo and deployable)
- Change summarised in commit message: what changed and why
For Claude models, also record: model version pinned, temperature, max_tokens, and any top_p / top_k settings — these affect output distribution as much as the prompt text.