Prompt Engineer Toolkit
Overview
Use this skill to move prompts from ad-hoc drafts to production assets with repeatable testing, versioning, and regression safety. It emphasizes measurable quality over intuition. Apply it when launching a new LLM feature that needs reliable outputs, when prompt quality degrades after model or instruction changes, when multiple team members edit prompts and need history/diffs, when you need evidence-based prompt choice for production rollout, or when you want consistent prompt governance across environments.
Core Capabilities
- A/B prompt evaluation against structured test cases
- Quantitative scoring for adherence, relevance, and safety checks
- Prompt version tracking with immutable history and changelog
- Prompt diffs to review behavior-impacting edits
- Reusable prompt templates and selection guidance
- Regression-friendly workflows for model/prompt updates
Key Workflows
1. Run Prompt A/B Test
Prepare JSON test cases and run:
python3 scripts/prompt_tester.py \
--prompt-a-file prompts/a.txt \
--prompt-b-file prompts/b.txt \
--cases-file testcases.json \
--runner-cmd 'my-llm-cli --prompt {prompt} --input {input}' \
--format text
Input can also come from stdin/--input JSON payload.
2. Choose Winner With Evidence
The tester scores outputs per case and aggregates:
- expected content coverage
- forbidden content violations
- regex/format compliance
- output length sanity
Use the higher-scoring prompt as candidate baseline, then run regression suite.
3. Version Prompts
# Add version
python3 scripts/prompt_versioner.py add \
--name support_classifier \
--prompt-file prompts/support_v3.txt \
--author alice
# Diff versions
python3 scripts/prompt_versioner.py diff --name support_classifier --from-version 2 --to-version 3
# Changelog
python3 scripts/prompt_versioner.py changelog --name support_classifier
4. Regression Loop
- Store baseline version.
- Propose prompt edits.
- Re-run A/B test.
- Promote only if score and safety constraints improve.
Script Interfaces
python3 scripts/prompt_tester.py --help
- Reads prompts/cases from stdin or
--input
- Optional external runner command
- Emits text or JSON metrics
python3 scripts/prompt_versioner.py --help
- Manages prompt history (
add, list, diff, changelog)
- Stores metadata and content snapshots locally
Pitfalls, Best Practices & Review Checklist
Avoid these mistakes:
- Picking prompts from single-case outputs — use a realistic, edge-case-rich test suite.
- Changing prompt and model simultaneously — always isolate variables.
- Missing
must_not_contain (forbidden-content) checks in evaluation criteria.
- Editing prompts without version metadata, author, or change rationale.
- Skipping semantic diffs before deploying a new prompt version.
- Optimizing one benchmark while harming edge cases — track the full suite.
- Model swap without rerunning the baseline A/B suite.
Before promoting any prompt, confirm:
References
- references/prompt-templates.md — 6 production marketing templates (ad copy, email sequence, social repurposing, landing sections, SEO meta, brand-voice rewrite) plus generic building blocks; each written to be graded by
prompt_tester.py
- references/technique-guide.md — technique-selection table for marketing tasks + the LLM-governance stack for marketing teams (claim discipline, disclosure rules, data boundaries, human-review gates)
- references/evaluation-rubric.md — mechanical scoring weights, acceptance gates, marketing quality dimensions, test-suite design, and eval anti-patterns
- README.md
Evaluation Design
Each test case should define:
input: realistic production-like input
expected_contains: required markers/content
forbidden_contains: disallowed phrases or unsafe content
expected_regex: required structural patterns
This enables deterministic grading across prompt variants.
Versioning Policy
- Use semantic prompt identifiers per feature (
support_classifier, ad_copy_shortform).
- Record author + change note for every revision.
- Never overwrite historical versions.
- Diff before promoting a new prompt to production.
Rollout Strategy
- Create baseline prompt version.
- Propose candidate prompt.
- Run A/B suite against same cases.
- Promote only if winner improves average and keeps violation count at zero.
- Track post-release feedback and feed new failure cases back into test suite.
1---2name: prompt-engineer-toolkit3description: "Turns marketing prompts into tested, versioned production assets: A/B prompt evaluation against structured test cases, immutable prompt version history with diffs, ready-to-use marketing prompt templates (ad copy, email campaigns, social posts, landing pages, SEO meta), and an LLM-governance playbook for marketing teams (claim discipline, disclosure rules, human-review gates). Use when a marketing team relies on AI-generated content and needs prompt quality to be measurable and safe — or when the user mentions 'prompt engineering,' 'improve my prompts,' 'prompt templates,' 'prompt versioning,' 'AI content workflow,' or 'AI governance for marketing.'"4license: MIT5metadata:6 source: https://skillmd.com/skills/bestagentkits/prompt-engineer-toolkit7 version: 1.0.08 author: Alireza Rezvani9 category: marketing10 updated: 2026-03-0611---1213# Prompt Engineer Toolkit1415## Overview1617Use this skill to move prompts from ad-hoc drafts to production assets with repeatable testing, versioning, and regression safety. It emphasizes measurable quality over intuition. Apply it when launching a new LLM feature that needs reliable outputs, when prompt quality degrades after model or instruction changes, when multiple team members edit prompts and need history/diffs, when you need evidence-based prompt choice for production rollout, or when you want consistent prompt governance across environments.1819## Core Capabilities2021- A/B prompt evaluation against structured test cases22- Quantitative scoring for adherence, relevance, and safety checks23- Prompt version tracking with immutable history and changelog24- Prompt diffs to review behavior-impacting edits25- Reusable prompt templates and selection guidance26- Regression-friendly workflows for model/prompt updates2728## Key Workflows2930### 1. Run Prompt A/B Test3132Prepare JSON test cases and run:3334```bash35python3 scripts/prompt_tester.py \36 --prompt-a-file prompts/a.txt \37 --prompt-b-file prompts/b.txt \38 --cases-file testcases.json \39 --runner-cmd 'my-llm-cli --prompt {prompt} --input {input}' \40 --format text41```4243Input can also come from stdin/`--input` JSON payload.4445### 2. Choose Winner With Evidence4647The tester scores outputs per case and aggregates:4849- expected content coverage50- forbidden content violations51- regex/format compliance52- output length sanity5354Use the higher-scoring prompt as candidate baseline, then run regression suite.5556### 3. Version Prompts5758```bash59# Add version60python3 scripts/prompt_versioner.py add \61 --name support_classifier \62 --prompt-file prompts/support_v3.txt \63 --author alice6465# Diff versions66python3 scripts/prompt_versioner.py diff --name support_classifier --from-version 2 --to-version 36768# Changelog69python3 scripts/prompt_versioner.py changelog --name support_classifier70```7172### 4. Regression Loop73741. Store baseline version.752. Propose prompt edits.763. Re-run A/B test.774. Promote only if score and safety constraints improve.7879## Script Interfaces8081- `python3 scripts/prompt_tester.py --help`82 - Reads prompts/cases from stdin or `--input`83 - Optional external runner command84 - Emits text or JSON metrics85- `python3 scripts/prompt_versioner.py --help`86 - Manages prompt history (`add`, `list`, `diff`, `changelog`)87 - Stores metadata and content snapshots locally8889## Pitfalls, Best Practices & Review Checklist9091**Avoid these mistakes:**921. Picking prompts from single-case outputs — use a realistic, edge-case-rich test suite.932. Changing prompt and model simultaneously — always isolate variables.943. Missing `must_not_contain` (forbidden-content) checks in evaluation criteria.954. Editing prompts without version metadata, author, or change rationale.965. Skipping semantic diffs before deploying a new prompt version.976. Optimizing one benchmark while harming edge cases — track the full suite.987. Model swap without rerunning the baseline A/B suite.99100**Before promoting any prompt, confirm:**101- [ ] Task intent is explicit and unambiguous.102- [ ] Output schema/format is explicit.103- [ ] Safety and exclusion constraints are explicit.104- [ ] No contradictory instructions.105- [ ] No unnecessary verbosity tokens.106- [ ] A/B score improves and violation count stays at zero.107108## References109110- [references/prompt-templates.md](references/prompt-templates.md) — 6 production marketing templates (ad copy, email sequence, social repurposing, landing sections, SEO meta, brand-voice rewrite) plus generic building blocks; each written to be graded by `prompt_tester.py`111- [references/technique-guide.md](references/technique-guide.md) — technique-selection table for marketing tasks + the LLM-governance stack for marketing teams (claim discipline, disclosure rules, data boundaries, human-review gates)112- [references/evaluation-rubric.md](references/evaluation-rubric.md) — mechanical scoring weights, acceptance gates, marketing quality dimensions, test-suite design, and eval anti-patterns113- [README.md](README.md)114115## Evaluation Design116117Each test case should define:118119- `input`: realistic production-like input120- `expected_contains`: required markers/content121- `forbidden_contains`: disallowed phrases or unsafe content122- `expected_regex`: required structural patterns123124This enables deterministic grading across prompt variants.125126## Versioning Policy127128- Use semantic prompt identifiers per feature (`support_classifier`, `ad_copy_shortform`).129- Record author + change note for every revision.130- Never overwrite historical versions.131- Diff before promoting a new prompt to production.132133## Rollout Strategy1341351. Create baseline prompt version.1362. Propose candidate prompt.1373. Run A/B suite against same cases.1384. Promote only if winner improves average and keeps violation count at zero.1395. Track post-release feedback and feed new failure cases back into test suite.