event4u-app prompt-engineering-patterns
Production patterns for LLM prompts: few-shot, chain-of-thought, system-prompt design, templating, self-verification. Distinct surface from sibling skills:
prompt-optimizer— polishes a single end-user prompt for ChatGPT / Claude / Gemini.refine-prompt— refines a free-form work prompt into engine-ready acceptance criteria.- This skill — designs prompts that ship inside an application that calls an LLM at runtime.
When to use
- Designing the system prompt for a new LLM-powered feature.
- Building a few-shot template with dynamic example selection.
- Adding chain-of-thought reasoning to a low-accuracy prompt.
- Reviewing a prompt diff in production code.
- Diagnosing inconsistent LLM outputs that look like prompt drift.
Do NOT use when:
- Polishing a one-off prompt for a chat session — route to
prompt-optimizer. - Turning a Jira ticket into engine input — route to
refine-prompt. - Tuning a model's weights — this skill is prompt-only, not fine-tuning.
Decision framework
Step 1 — Pick the prompt level (progressive disclosure)
Start at Level 1; only escalate when measurement says you must.
Level 1 Direct instruction "Summarize this article."
Level 2 + constraints (length, format, focus) "...in 3 bullets, key findings only."
Level 3 + reasoning scaffold "Read first, identify findings, then summarize."
Level 4 + few-shot examples "Like these examples: ..."
Level 5 + self-verification step "...then check answer against criteria; revise if fails."
Escalating without evidence is over-engineering. Each level adds tokens, latency, and a maintenance surface.
Step 2 — Structure the prompt
Fixed instruction hierarchy — every production prompt fills these slots in order:
[System context] role, expertise, constraints, safety
[Task instruction] what to do, in one sentence
[Examples] few-shot demonstrations (optional)
[Input data] the user-supplied content
[Output format] schema, length, citation rules
Stable slots (system, task, format) belong in cached prompt prefixes; volatile slots (examples, input) belong in the per-call portion.
Step 3 — Pick the few-shot strategy
Examples are uniform and small (< 20) → embed all of them; deterministic.
Examples are large or diverse → semantic-similarity retrieval per call.
Edge cases dominate → diversity-sampled examples (cluster + pick one per cluster).
Token budget tight → fewer, higher-quality examples beats many mediocre.
Examples drift with the data → regenerate from a labeled corpus on a schedule, not hand-edited.
Bad examples are worse than no examples — the model imitates structure.
Step 4 — Add chain-of-thought ONLY when measured
CoT improves accuracy on multi-step reasoning, hurts on classification and lookup. Decision rule:
Task is multi-step / arithmetic / multi-hop → add CoT (zero-shot "let's think step by step", or few-shot CoT).
Task is single-step extraction / classify → CoT adds tokens without lift; skip.
You haven't measured → measure first, decide second.
Self-consistency needed (high-stakes answers) → sample N reasoning paths, majority vote.
Step 5 — Build error recovery into the prompt
Production prompts handle their own failure cases:
- Specify the explicit "I don't know" output (don't let the model invent).
- Require a confidence indicator when downstream code needs to gate.
- Define the format for "missing information" so callers can branch.
- For self-verification: specify the criteria, then the revision rule.
Step 6 — Treat prompts as code
- Version every prompt (file + git, not a wiki page).
- Test on a frozen evaluation set before shipping changes.
- Track P50 / P95 latency, token usage, accuracy, success rate per version.
- A/B test prompt variants behind a flag; never edit a live prompt without a rollback path.
Procedure: Apply to a new LLM feature
- Inspect the existing prompt (if any) and the eval set; verify a success metric exists (accuracy / consistency / latency / token cost) — refuse to design without it.
- Draft Level-1 prompt (Step 1) and measure on the eval set.
- Escalate one level at a time (Step 1) until metric is met or budget runs out.
- Lock the structure (Step 2), choose few-shot strategy (Step 3), decide CoT (Step 4).
- Add error-recovery clauses (Step 5).
- Commit prompt + eval results + chosen version (Step 6); cite this skill.
Output format
- Prompt-spec table: slot · content · stable-vs-volatile · cached-vs-per-call.
- Eval results table: prompt-version · metric · delta-vs-previous.
- Failure-mode list: trigger · prompt clause that handles it.
Gotcha
- Few-shot examples leak the model's style — examples that include hedging produce hedging.
- "Let's think step by step" works zero-shot on capable models, fails on smaller models without exemplar reasoning traces.
- Self-consistency (N samples + vote) multiplies cost by N — only on high-stakes paths.
- Cached prompt prefixes only cache when byte-identical — a single reformat busts the cache.
- Prompts that drift across model versions silently regress accuracy when the provider rolls a model update; pin model version OR re-run eval per release.
Do NOT
- Do NOT escalate to Level 4 / 5 before measuring at lower levels.
- Do NOT mix few-shot examples from different tasks; the model averages them.
- Do NOT add CoT to single-step classification — it hurts.
- Do NOT hand-edit production prompts without versioning + eval.
- Do NOT echo secrets or PII into the prompt — they end up in provider logs.
Auto-trigger keywords
- prompt engineering
- few-shot learning
- chain-of-thought
- system prompt design
- prompt template
- LLM prompt versioning
- prompt evaluation
Provenance
- Adopted from: an external reference (internal provenance, redacted) — restructured into a decision-framework shape; source-specific vendor Python snippets dropped.
- Cross-linked:
prompt-optimizer,refine-prompt,mcp-builder,async-python-patterns. - Provenance registry:
agents/settings/contexts/skills-provenance.yml(entry:prompt-engineering-patterns). - Iron-Law floor:
verify-before-complete,skill-quality,non-destructive-by-default.