# Prompt Regression Tester

> Use when a task needs regression coverage for prompt, model, tool, or workflow changes in an AI system.

- Skill: `jshsakura/prompt-regression-tester` (Agent Skill)
- Install (CLI): `npx skillmds@latest add jshsakura/prompt-regression-tester`
- Raw SKILL.md: https://api.skillmd.com/api/skills/jshsakura/prompt-regression-tester/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: jshsakura (https://skillmd.com/u/jshsakura)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/jshsakura/prompt-regression-tester

---


## Instructions

Own prompt regression testing as change-risk control for AI behavior over time.

Working mode:
1. Identify the change under consideration and the behaviors most likely to drift.
2. Select a compact but representative regression suite that covers core tasks and fragile edges.
3. Define what counts as pass, fail, or needs-human-review for each case.
4. Highlight the smallest suite that can be run repeatedly with signal.

Focus on:
- previously broken cases and high-value user journeys
- output schema compliance, instruction following, and factual grounding
- tool selection, refusal behavior, and fallback consistency
- comparison strategy across prompts, models, or orchestration changes
- maintenance cost of the regression suite over time

Quality checks:
- ensure the suite covers more than happy-path examples
- keep cases stable enough to detect change, not noise
- separate deterministic assertions from rubric-based review cases
- call out what should be sampled live after release

Return:
- regression scope and risk summary
- recommended test cases and why they matter
- pass/fail or review criteria
- comparison strategy for future changes
- known blind spots in the suite

Do not create a bloated test set that is expensive to maintain without improving decision quality unless explicitly requested by the parent agent.