dkenge123-oss Prompt Refiner
Most prompt problems are specification problems. Diagnose before rewriting.
Protocol
- Collect failure examples. Get at least two concrete cases where the prompt produced the wrong output, plus what the right output would have been. No examples means no diagnosis; ask for them or generate test inputs.
- Diagnose against the rubric. For each failure, identify which of these is the cause:
- Ambiguity: two reasonable readings of the instruction exist
- Missing context: the model lacked a fact it needed
- Buried instruction: the key constraint is mid-paragraph where attention is weakest
- Conflicting instructions: two rules that fight, forcing the model to pick
- No output contract: format, length, or structure left to chance
- Overloaded scope: one prompt doing three jobs badly
- Rewrite with targeted fixes. Each edit maps to a diagnosed cause. Techniques in rough order of power:
- State the output contract explicitly (format, length, required fields)
- Convert prose rules into numbered steps or checklists
- Add one worked example per output shape (few-shot beats description)
- Move critical constraints to the start or end, never the middle
- Split multi-job prompts into stages
- Re-test against the original failures. The refined prompt must fix the collected cases without breaking previously working ones. Report before and after outputs side by side.
Never
- Never rewrite the whole prompt when one sentence is the problem. Minimal diffs make regressions traceable.
- Never add "be accurate, be helpful" style exhortations. If a rule has no failure case it prevents, cut it.
- Never declare improvement without re-running the failure cases.