Satisfaction rate
The bundled script reads the transcripts already on this machine and rates every request the user typed by their next move. A correction ("no, that's wrong"), a complaint, a near-identical retry, or an interrupt counts as unsatisfied. Moving on or saying thanks counts as satisfied. It compares a recent window with the window before it. Nothing is sent anywhere, and it needs only Python 3.9+.
1. Run it
python3 ${CLAUDE_SKILL_DIR}/scripts/run.py --html /tmp/satisfaction-rate.html $ARGUMENTS
Defaults: the current window is the last 2 days and the baseline is the 14 days before that, broken down by model. Useful flags:
--model opus: only exchanges answered by a model whose id contains "opus"--current-days 3 --baseline-days 21: change the windows--by source: break down by tool instead of model--min-rated 20: lower the sample size needed for a verdict (default 30)--json: machine-readable output
If the user named a model or a time range, translate it into these flags.
2. Report it honestly
- Lead with the verdict for the model the user asked about, or for
all. Quote both rates, bothnvalues, the change in points, and the 95% interval. - Say "decline" only when the tool's verdict says decline. "No detectable change" and "insufficient data" are answers too. Do not round them up into a decline.
- Repeat every
caution:line in plain words. When the tool mix or prompt size shifted, the work changed, and harder tasks can lower satisfaction without any model change. - If a window has too few rated exchanges, say how many it has and suggest a longer
--current-days. - Mention which unsatisfied outcomes moved (corrections, frustration, retries, interrupts), from the "What drove unsatisfied exchanges" lines.
- Close with one sentence of context: a lower satisfaction rate shows the user was less satisfied, not that the provider changed the model. Re-running a fixed set of prompts on a schedule is the way to test the model itself.
- Offer to open the chart with
open /tmp/satisfaction-rate.html.
Never paste transcript content into the reply beyond what the report prints.
Notes
- Claude Code deletes transcripts older than
cleanupPeriodDays(30 by default), so the baseline can only reach back that far. - The phrase lists are English-only. See
scripts/satisfaction_rate/signals.py.