# Health Score Designer

> When the user wants to design, weight, calibrate, backtest, audit, or fix a customer health score. Also use when the user mentions 'health score is garbage', 'nobody trusts it', 'score that actually predicts', 'design a health score', 'build a customer health score', 'our health score is broken', 'nobody trusts the health score', 'what should go into our health score', 'health score weights', 'green accounts keep churning', 'is our health score any good', 'backtest the health score', 'calibrate our health score', 'red yellow green thresholds', 'the score isn't predicting anything', 'audit our scoring model', or 'rescore the book'. Use this whenever someone is deciding what makes an account healthy, or arguing about a score's inputs, even if they never say 'health score' — a request for 'one number that tells me who to work on' is this skill. For scoring one account today, see churn-risk. For the data inventory the score is built on, see cs-context. For per-account renewal probability, see renewal-forecast.

- Skill: `gaintrace/health-score-designer` (Agent Skill, multi-file: 10 files)
- Install (CLI): `npx skillmds@latest add gaintrace/health-score-designer`
- Raw SKILL.md: https://api.skillmd.com/api/skills/gaintrace/health-score-designer/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- License: MIT
- Author: gaintrace (https://skillmd.com/u/gaintrace)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/gaintrace/health-score-designer

---


# Health Score Designer

You own the health score as a **measurement instrument**, not a dashboard widget. The standard is a score a CFO
would accept as an input to the renewal forecast: it predicts a named commercial event over a named horizon, and it
has been checked against what actually happened. The rookie version is a workshop — eight people pick eight
dimensions, argue weights until they sum to 100, ship it, and never look again, so the score becomes an average of
mediocrity that lands every account between 60 and 75, ranks nothing, and gets ignored. **73% of customer and
post-sales leaders say their health score does not reliably predict churn** [2025 Customer Revenue Leadership
Study, Pavilion / 6sense, ~800 customer and post-sales leaders — self-reported `[M]`].

The elite version does four things the rookie version never does: writes the prediction as a **falsifiable
sentence** before choosing a single input; derives weights from **observed renewal outcomes**; sets the red
threshold from **CSM capacity** rather than F1; and ships **reason codes** so the number arrives attached to a play.
Read `../cs-context/references/evidence-standard.md` first — a spec quoting an unsourced number as a threshold
starts the "where did 70 come from?" argument that runs for three years.

## Before Starting

1. **Read `.agents/cs-context.md`** (fallback `.claude/cs-context.md`). If absent, run `cs-context`. §2 (commercial
   model, notice period), §5 (activation event), §6 (existing score and its trust level), §9 (sources) and §12
   (coverage) are load-bearing. **Never ask for anything that file already answers** — ARR, renewal dates, notice
   period, segment boundaries, fiscal year, source inventory. Asking tells the user the skill did not read it.

2. **Take the data in whatever shape it arrives.** Do not specify a format and do not ask for a clean export. This
   skill accepts CSV, TSV, XLSX, JSON, NDJSON, warehouse query results, pasted text, call transcripts, a described
   screenshot — or no file at all, just answers to the questions in step 3.
   - **Run `../cs-context/scripts/ingest.py` first on every supplied file.** It sniffs encoding and delimiter, finds
     the real header row beneath the three title rows a CRM export puts above it, maps columns onto the canonical
     schema with a confidence per column, normalises dates, money-stored-as-text and booleans, resolves accounts
     across files, and reports the join rate.
   - **Confirm every column mapping below 0.80 confidence before using those numbers.** A column mapped
     `renewal_date → contract_start_date` mis-dates the opt-out deadline for every row, and the weight derived from
     it will not look wrong.
   - **Degrade, never refuse.** Partial data produces a partial spec with a coverage figure and a confidence cap —
     never an error, and never a request for better data first; the sparse variant (Step 3) exists for this. And
     **never assume an export is complete or current** — ask for the as-of date, print it in the header, and treat a
     file without one as stale until told otherwise.

3. **Ask up to four questions, once, tappably — then run unattended.** Use `AskUserQuestion` with every applicable
   question in a **single batch**; never drip-feed. Skip any question `cs-context` or the prompt already answers.

| Header | Question | Options — recommended first, each with what it changes |
| --- | --- | --- |
| `Purpose` | What is this score for? | **Triage — who do I work on this week (Recommended)** · threshold set by CSM capacity, bands only, no probabilities needed — **Forecast** · demands a fitted calibration map and a backtest before any number ships — **Exec reporting** · absolute scales only, no cohort percentiles (they are zero-sum) — **Compensation** · say plainly this guarantees gaming, then build the Step 11 controls |
| `Event` | What counts as the bad outcome? | **Non-renewal OR downsell ≥15% of ARR (Recommended)** · catches partial churn, which is most of the lost dollars — **Logo churn only** · simplest label, misses shrinkage entirely — **Gross ARR churn** · dollar-weighted, needs more events to fit — **Consumption shortfall** · for usage-based contracts, where churn shows as under-burn not cancellation |
| `Horizon` | How far ahead must it warn? | **120 days to the opt-out deadline (Recommended)** · enough time for a save motion to change the outcome — **90 days** · tighter, higher AUC, less room to act — **180 days** · maximum lead time; expect weaker discrimination and more false reds |
| `Model` | How do these customers buy? *(skip if `cs-context` §2 answers it)* | **Annual contracts with notice periods (Recommended)** · enterprise profile, usage 35 / commercial 15 — **Self-serve monthly** · PLG profile, usage 45 / relationship 5; billing rises because involuntary churn is material — **Usage-based / consumption** · commitment pacing replaces licence utilisation entirely — **Mixed** · profile resolved per account from its contract record |

4. **Never block on an answer, and never guess one.** Every missing input resolves exactly one of three ways —
   **read it** (derive it from the data or `cs-context`, showing the derivation), **ask it** (step 3, only where two
   likely answers produce a materially different spec), or **mark it** (`UNKNOWN — requires <source>` plus a
   confidence cap). There is no fourth way: a plausible substituted value becomes a fabricated one the moment someone
   repeats it. If the questions go unanswered, proceed on the recommended defaults — *label = non-renewal or downsell
   ≥15%, H = 120 days to the opt-out deadline, unit = subscription, population = tenure >120 days, enterprise weight
   profile, red threshold at the worst decile* — state them in one line at the top of the output and record each in
   the **Assumption Register**. **Capacity is read, not asked:** derive it from `coverage-and-capacity` or
   `cs-context` §3; only if neither has it, default to the worst decile and register that.

5. **Detect data state before promising anything.** Four facts decide which mode is legitimate. **Is score history
   stored as a time series** rather than overwritten in place? If not, Backtest and Calibrate are closed and fixing
   this is priority zero — nothing here can ever be validated without it and it cannot be recovered retroactively.
   **Are there ≥100 completed renewal outcomes in the last 18 months?** If not, weights are expert-elicited v0
   against proxy labels. **Is `churn_event.decision_date` captured, not only `effective_date`?** If not, say so — a
   model trained on `effective_date` largely memorises the notice period (`R24`). **Is product telemetry joined to
   `account_id` at ≥80%?** If not, build the sparse model (Step 3) with the gap printed rather than papered over.

6. **If a score is already in production, run Audit first.** Rebuilding before diagnosing throws away your one real
   asset: a history of the current score's errors.

## How This Skill Works

| Mode | Trigger | Produces |
| --- | --- | --- |
| **Design v0** | No score, or an approved rebuild | Specification v0 + instrumentation gap list |
| **Calibrate v1** | Stored history + ≥100 outcomes | Re-derived weights, re-cut bands, score→probability map |
| **Backtest** | "Is our score any good?" | Lift, ARR capture, AUC, Brier, reliability, per segment |
| **Audit** | "Nobody trusts it" / "green accounts churn" | 30-point checklist + the *does-anyone-act-on-it* test |
| **Segment split** · **Rescore & migrate** | One global model under-serving both ends of the book · shipping a new version | Per-segment parameter table and reconciliation rule · migration matrix, comms plan and champion/challenger schedule |

**Brief by default, Full on request.** Brief is ≤20 lines and is not a summary written after Full — it is the answer,
written first: the verdict, the number that proves it, the one change with the largest effect on discrimination with
an owner and date, confidence in three words, and the falsifier. Emit Full when asked, when it goes to a board, a CFO
or a scoring committee, or when someone will challenge it. Brief obeys every evidence rule and drops only the
**display** of the reasoning.

**The rules this skill enforces**, from `../cs-context/references/operating-rules.md` — enforced in the output, not
merely mentioned, and any deviation is stated with its rule number, the circumstance, and what will be watched:

| Rule | Enforced how |
| --- | --- |
| **R1 · The Opt-Out Calendar** | The horizon is measured to `renewal_date − notice_period_days`. The renewal date never appears as a deadline, in the spec or in a backtest snapshot |
| **R2 · Decisions Beat Indicators** | The override caps in Step 6. A weighted 71 does not survive an auto-renew flag that flipped |
| **R8 · The Health Gate** | The Green band's expansion play is gated on the health floor, and the gate is printed in the band table |
| **R13 · The Capacity Truth** | The red threshold is derived from usable save hours (Step 9), never from F1 |
| **R18 · The Firewall** | This skill emits **no customer-facing text**. A health score, band or ARR-at-risk figure never reaches a customer in any wording; every page of the spec is marked Internal |
| **R22 · Ordering Before Probability** | Bands and deciles only, until a calibration map is fitted on held-out data and its backtest is cited |
| **R23 · The Coverage Cap** | All seven families printed including the missing ones; confidence never exceeds coverage |
| **R24 · Label the Decision, Not the Event** | Labels dated at `decision_date`. Training on `effective_date` teaches the model your notice period |

**Know the business model before scoring.** Read `../cs-context/references/business-model-profiles.md` and resolve the
profile before Step 1. It decides which standard practices **do not apply** — licence utilisation on a consumption
business, QBR completion on a PLG business and sponsor state on a tech-touch book are all manufactured risk, and
recommending them is the most recognisable form of generic output. Weights per model: `model-archetypes.md` §3.

Run sequence: **prediction sentence → archetype → dimensions → scoring functions → weights → non-compensatory
layer → segmentation → backtest → capacity thresholds → explainability → governance.** Two conventions: **health
space is 0–100, higher is better** — `churn-risk` scores risk, where higher is worse, and `health = 100 − risk` is
not a safe conversion, so state the direction on every artifact; and **override rules are floors in risk space and
caps in health space**, written here as caps.

## Step 1 — Write the falsifiable sentence first

Before any input is chosen, complete this. If you cannot, you are building a dashboard.

> An account scoring **X** on date *d* has probability **p** of **<named event>** at its next renewal decision
> within horizon **H**, at the **<unit>** grain, over population **<P>**.

| Decision | Options | Failure if unstated |
| --- | --- | --- |
| **Event (label)** | logo churn · gross ARR churn · downsell ≥15% ARR · non-renewal-or-downsell | Expansion signals cancel churn signals and the score collapses to the mean |
| **Horizon H** | 90 / 120 / 180 days to the **decision date** | The score is graded against events it never had a chance to see |
| **Unit** | account · subscription · product line · parent group | A multi-product account goes red for one bad product; the renewal is mis-forecast |
| **Population** | tenure ≥N days · segment · touch model | Onboarding accounts with no baseline get scored on trend and produce noise |

**Use the decision date, never the renewal date** (`R1`, `R24`). It lands at `opt_out_deadline = renewal_date −
notice_period_days`, earlier still if the customer's budget locks before their fiscal year; a horizon anchored to the
renewal date is late by exactly the notice period. Defaults as in Before Starting step 4, and track logo and ARR
outcomes as **separate labels** — catching 80% of churned logos and 30% of churned dollars is a failure.

## Step 2 — Choose the archetype from data maturity, not ambition

**Default: rubric + override caps + a published velocity, moving to hybrid after one full renewal cycle of stored
history.** Choose from what you can populate for 90% of the book every day, not from ambition. No stored history →
rubric, and fix history storage first. Under 100 outcomes → rubric with v0 weights. Under 300 outcomes, or no named
owner to re-fit forever → rubric with outcome-derived weights. ML propensity only when all six preconditions hold;
a hybrid always publishes its reconciliation rule, or you get two sources of truth and trust in neither. GitLab
runs the hybrid in production — a PROVE rubric alongside separate propensity models — after abandoning a pure black
box because it "was not easy to understand the calculation… and was not action-oriented" [GitLab Handbook,
*Customer Health Scoring*, published production config].

All seven archetypes with their minimum data, failure modes, the decision tree, the six ML preconditions and the
migration path: `references/model-archetypes.md` — read it before defending the choice to a sceptical exec.

## Step 3 — Select dimensions from the seven signal families

Dimensions come from the library's seven fixed families — do not invent parallel names. Value realisation and ROI
score **inside Product usage & adoption**; success plans and executive business reviews score **inside Relationship
& engagement**. Name the family on every row.

| # | Family | Candidate dimensions | Ent. | PLG | Sparse |
| --- | --- | --- | --- | --- | --- |
| 1 | **Product usage & adoption** | Licence utilisation · power-user retention · value-event trend vs own baseline · feature breadth · TTFV met · verified outcomes closed | 35 | 45 | **0 — unmeasurable** |
| 2 | **Commercial & contract** | Auto-renew state · contract trajectory · discount depth vs segment median · seat trajectory · procurement re-opened | 15 | 10 | 20 |
| 3 | **Relationship & engagement** | Sponsor state · persona-covered multithreading · meeting-cadence recency · EBR completion · success-plan quality | 20 | 5 | 30 |
| 4 | **Support & reliability** | Tickets per 100 seats (U-curve) · open P1s · reopen rate · age of oldest open ticket | 10 | 10 | 20 |
| 5 | **Sentiment & VoC** | CSM sentiment (R/Y/G + written justification) · CSAT trend on resolved tickets · NPS with response rate | 10 | 5 | 10 |
| 6 | **Billing & payment** | Days-late trend · payment failures · payment-method status · plan-limit headroom | 5 | 20 | 15 |
| 7 | **Firmographic & external** | Funding / layoff / M&A events · headcount trend · leadership change | 5 | 5 | 5 |

Weights are `[DESIGN]` starting points seeded from published configurations, **not measured benchmarks**, and exist
to be replaced by Step 5 (consumption profile: `references/model-archetypes.md` §3). **A dimension must pass all
four axes below or it does not belong.**

| Axis | Test | Reject if |
| --- | --- | --- |
| **Predictiveness** | Univariate AUC ≥0.58, or ≥1.3× top-decile lift, in *your* backtest | It separates nothing |
| **Lead time** | Days before the decision date at which it moves | It only moves inside 30 days — a confirmation device, not a warning |
| **Actionability** | If it turns red, name the play, the owner and the SLA | You cannot. It belongs on a report, not in the score |
| **Integrity** | Can a CSM or the customer move it without changing reality? | Yes, with no control (Step 11) |

Actionability is the axis people skip and the one that matters most: "industry" predicts churn and is entirely
unactionable, so it belongs in segmentation, not the score. **Weight hygiene**, verified programmatically after
every edit: **5–8 dimensions**, nothing below **5%**, nothing above **35%** in a composite of ≥5, sums to 100 at
every tier, **≥50% of weight on leading dimensions**. The floor is arithmetic, not taste — a dimension at 5% moves
the composite five points across its whole range, less than one band width, so it cannot change a decision.

**Sparse model (CRM + support + billing, no telemetry).** Product usage prints as `❌ Missing` at weight 0 and is
**not** redistributed — silent redistribution manufactures a false green (Step 6). Expect AUC **0.62–0.72** `[P]`,
report deciles not probabilities, lean harder on override caps (which need no calibration to be useful), and carry a
named instrumentation roadmap: login/MAU joined to `account_id` → 5–10 named value events → seat activation →
entitlement vs use (`references/model-archetypes.md` §6).

## Step 4 — Build a scoring function for each dimension

Raw values never enter a composite. Every dimension gets an explicit transform, a stated baseline and an NA rule. All
formulas, half-lives, guards, U-curves and per-dimension recipes: `references/scoring-functions.md` — read it while
writing §3 of the spec. **1. Score against the account's own baseline, not an absolute threshold** — a
weekly-cadence account that stays weekly is healthy; a daily account that drops to weekly is churning.

```
base   = median(weekly_value_events, weeks t-12 … t-5)     # median survives outages
recent = median(weekly_value_events, weeks t-3 … t)
trend  = (recent − base) / max(base, ε)
s_trend: ≥+0.10 → 100 | −0.10..+0.10 → 75 | −0.25..−0.10 → 50 | −0.50..−0.25 → 25 | <−0.50 → 0
Guard: require base ≥ 20 events, else return NA — not −100%.
```

**2. Recency decays exponentially**, one interpretable parameter: `s_recency = 100 × exp(−ln 2 × days_since_touch /
HL)`, HL = expected cadence in days, so one missed cadence scores exactly 50; half-lives per signal are in its §2.
**Never decay state** — auto-renew being off does not become less true after 60 days.

**3. Cohort-relative only where there is no natural scale, never for absolute reporting.** Percentile rank within
`segment × tenure_band × product_tier`, minimum cohort size 30 `[P]`. Cohort percentiles are **zero-sum** — half the
book is below median by construction, so portfolio health can never improve.

**4. Suppress noise before it fires** (its §5): the consecutive-period rule — fire only after N=2 consecutive
weekly periods below threshold, which cuts single-period false positives by roughly an order of magnitude `[P]`;
seasonality masks over the customer's low season, holidays, academic summer and retail freeze windows; and a
fleet-wide diff, because an event-taxonomy rename decays the whole book at once — if >15% of accounts move the same
direction by >5 points in a day, suppress and page CS Ops.

**Banded** transforms for anything a human acts on, **continuous** for anything feeding a model or a ranking — mixing
them undocumented is a top cause of "the score jumped 12 points and nobody knows why." **Zero support tickets is not
green:** U-curve the support dimension (0 → 60, healthy band → 100, heavy volume → 20) and normalise per 100 licensed
seats, or every large account is red.

## Step 5 — Set the weights from outcomes, not from a workshop

| Method | Procedure | Requires | Watch |
| --- | --- | --- | --- |
| **v0 expert** | Structured elicitation; force sum to 100; document the *rationale per dimension* so it can be falsified later | No labels yet | Weights encode politics. Timestamp them, mark provisional |
| **v1-lite lift** | Per dimension: loss rate in worst tercile ÷ loss rate in best tercile. Rank by lift, assign in proportion, clamp to the hygiene band | ≥100 outcomes | Univariate lift double-counts correlated dimensions — deduplicate within a family first |
| **v1 outcome-derived** | Fit `logit(P(negative)) = β₀ + Σβᵢxᵢ` on point-in-time snapshots; set `wᵢ ∝ \|βᵢ\| · σ(xᵢ)`, round to human numbers | ≥300 outcomes, ≥50 negatives | Usage metrics are collinear and will flip signs. Use L1/L2, or group into pillars first |
| **v2 platform recommender** | Your CS platform may ship one. Demand four things in writing first: minimum sample size, required overlap between scorecard history and renewal outcomes, staleness policy, and the definition of "prediction power" it optimises | Whatever it states | Validate against your own backtest before it drives a play. A recommender you cannot interrogate is a black box with a friendlier UI |

**Sample-size honesty.** The classic rule is **10–20 churn events per predictor** [Peduzzi et al., 1996 `[A]`] — 8
predictors means 80–160 *events*, not 80 accounts. Riley et al. (*Statistics in Medicine*, 2019) showed
events-per-variable rules of thumb are unreliable and derived closed-form minimums accounting for prevalence and
shrinkage `[A]`. Below the floor, ship v0 and say so. **Cold start:** ship the rubric, instrument everything, **record
score history from day one**, and validate against proxy labels (downsell, escalations, sponsor departures,
auto-renew disabled) until one renewal cycle closes. **Then freeze weights for a full cycle before judging them** —
re-weighting monthly makes backtesting impossible, because every backtest is against a different score.

## Step 6 — Add the non-compensatory layer

A weighted sum lets a 100 on Support offset a 0 on Sponsor, but real churn is usually one fatal condition, not an
average of mediocrity. Three fixes, cheapest first: publish sub-scores as first-class (never render the composite
without its 4–6 pillars); a **worst-pillar rollup** — the composite takes the band of its weakest pillar, or
`0.7 × weighted_avg + 0.3 × min(subscores)`; and override caps. **Defaults `[DESIGN]`** — each is a decision or a
structural fact, not an indicator, which is why an aggregate score must not wash it out (`R2`):

**Non-renewal notice served or auto-renew switched off → cap ≤29**, because that is the decision itself.
**Competitive evaluation or RFP confirmed → ≤49.** **Executive sponsor departed with no replacement named within 45
days → ≤49** — the contract has no owner on their side (`R3`). **Invoice >60 days overdue → ≤49**, because
non-payment precedes non-renewal. **CSM sentiment Red with written justification → ≤49**, GitLab's exact production
behaviour. **Licence utilisation <50% at >180 days tenure → ≤59**; partial churn has already happened. **Data
sufficiency under 70% of weight populated → suppress to "Insufficient Data"**, because an instrumentation failure is
an ops defect, not a customer state. Each cap needs an evidence source and an expiry in the spec
(`assets/score-spec-template.md` §4).

**The NA rule is the most consequential line in the spec.** The usual platform default redistributes a missing
dimension's weight across the survivors **in proportion to their existing weights** — silently handing the biggest
increase to whatever was already heaviest, a consequence GitLab documents in its own config. Choose explicitly
between (a) proportional redistribution capped at 20% of original weight, (b) impute neutral, (c) composite = NA,
(d) structural zero with the denominator printed (`references/scoring-functions.md` §6); and **never let missing
data produce a green**. **Staleness forces NA**, because a frozen green on a dead feed is the worst failure in the
catalogue — nothing about it looks wrong. GitLab's published policy: product usage → NA after **60 days** without
data; support → **30 days**; CSM sentiment stale at **90 days**, forced NA at **120 days** — they "prefer to show
nothing ('NA') over outdated data."

## Step 7 — Segment the model

Segment when the **relationship between a signal and the outcome differs**, not merely when the level differs. If SMB
and Enterprise both churn on usage decline at different absolute levels, you need different *thresholds*, not
different *models*.

The five splits that earn their cost — lifecycle stage, touch model, entitlement tier, ACV band and pricing model
— with the published GitLab and Benchmarkit anchors for each, are in `references/model-archetypes.md` §8. Read it
before adding a variant. The two that are almost always right: **NULL every trend measure for the first 30 days**
(onboarding accounts have no baseline), and **mark relationship dimensions N/A, never zero, for tech-touch**.

**Prefer one model with segment-parameterised thresholds over five separate models.** Ceiling: **3–5 variants
`[DESIGN]`** — each is a separate spec, backtest, band cut and governance review, so the fifth costs more than it
discriminates.

## Step 8 — Backtest against actual outcomes

Run `scripts/backtest.py` (worked example: `scripts/sample-scores.csv`). The eight-step protocol, the seven leakage
traps, the full metric suite and the false-green / false-red taxonomy are in `references/calibration.md` — read §1
and §2 before the first snapshot, and §6 before the autopsy.

The four that decide whether the result means anything: **snapshot point-in-time from the opt-out deadline**, at
T−180/120/90/60/30 · **rebuild every feature as-of that date from immutable event logs**, never current-state tables
· **split temporally**, never randomly · **exclude the leakage features** — churn reason, `renewal_status`,
cancellation-request ticket type, stage = Closed Lost, and CSM sentiment recorded after notice (GitLab's field
refreshes every two hours, so it is contaminated within hours of a churn conversation). Then beat three baselines —
constant base rate, single best feature, last quarter's model — compute per segment as well as overall, plot the
lead-time curve, and **autopsy 20 false-greens and 20 false-reds by hand**, which produces more improvement per hour
than any parameter search. Neslin et al. (*JMR*, 2006) found churn models suffered "very little decrease in
performance" on a database compiled three months after the calibration data `[A]`; if yours collapses beyond T−45,
suspect leakage in the short-horizon version.

**The five metrics that decide it** `[DESIGN]`, in reporting order — full suite, definitions and targets in
`references/calibration.md` §3: **base rate** in logos *and* ARR, computed per segment, because every other metric
is read against it · **top-decile lift**, ≥2.5× minimum and 3–4× good — if CS only works the worst 10%, how much
better than random? · **ARR capture @ decile**, ≥40% in the worst 10% and ≥65% in the worst 20%, because catching
80% of logos and 30% of dollars is a failure · **PR-AUC** at ≥2.5× the base rate, which is the metric to use
instead of ROC-AUC whenever churn is under 10% · and the **reliability curve**, monotone and near-diagonal, the
most persuasive chart a CCO will see and meaningful only once a map is fitted.

**Accuracy is banned as a headline metric.** At a 5% base rate, predicting nobody churns is 95% "accurate" and
worth nothing. **Turning the score into a probability**, only after this backtest: hold out a validation set used
for neither weights nor bands, bin into deciles, fit a monotone map — Platt under ~1,000 outcomes, isotonic above
`[P]`. Re-fit the map quarterly, the weights far less often. Until it exists, publish **bands and deciles, never a
percentage** (`R22`). Where `.agents/cs-calibration.json` exists, use its observed band rates and cite the sample
size (`../cs-context/scripts/calibrate.py`).

## Step 9 — Set thresholds from CSM capacity, not from F1

The red line is an operations decision (`R13`): you can only work the number of red accounts you have hours for, and
a red list longer than capacity trains everyone to ignore all of it.

```
capacity      = CSMs × usable_hours_for_saves ÷ hours_per_save_motion    # usable ≈ 60% of nominal
red_threshold = the score where count(accounts below it) ≈ quarterly save capacity
```

Worked: 8 CSMs × 4 usable hours/week × 13 weeks ÷ 10 hours per save motion ≈ **41 save motions per quarter**. That
number sets the threshold — not F1, not Youden's J, not a round number. At a realistic operating point you accept
roughly 38% precision and 48% recall, and **that is the correct answer, not a compromise**: working 400 accounts you
do not have hours for saves less revenue than working 200 you do. **Two floors `[P]`:** below ~25% precision CSM
credibility burns and alerts get ignored; below ~40% recall the score is not earning its build cost.
`scripts/backtest.py --capacity N` prints the threshold table from your own data and fires both warnings
(`references/calibration.md` §4 has the worked 2,000-account version).

Cut bands where the **observed** loss rate changes, not at round numbers. Published band cuts differ widely —
GitLab's public config uses Green 75–100 / Yellow 50–74 / Red 0–49 — which is the point: no cut is canonical,
because each is fitted to one book's distribution and one team's capacity. Derive yours, publish it, and never move
it mid-year without restating history.

## Step 10 — Ship explainability, or the score gets ignored

Every render must answer six questions with no analyst in the loop: what is the score · what was it 30 and 90 days
ago · which three dimensions moved it most · which single dimension is furthest below its band · what is the
recommended play · when was each input last refreshed.

**Attribute to the delta, not the level.** "Support is your lowest pillar" is true for half the book and tells nobody
anything; "meeting recency fell 22 points in 30 days and drove 60% of the drop" is a work order. For an ML model,
convert SHAP values into templated reason strings. **Publish velocity beside level:** a 62-and-falling account and a
62-and-rising account are different objects and one composite cannot express the difference.
`health_score_delta_30d` is a first-class field, and **green-but-falling ≥10 points in 30 days** is the most valuable,
most ignored cell in the model. **Every band owns a play** — action · owner · date · expected effect · success measure.

## Step 11 — Govern it

One named owner in CS Ops; a score owned by a committee drifts, breaks quietly and dies. Semantic versioning:
**major** = dimensions or label changed · **minor** = weights · **patch** = thresholds, and every version keeps its
spec and its backtest permanently. Champion/challenger: a new config runs in shadow for one full renewal cycle
before it drives a single play. Refresh: composite daily, **history retained forever**, calibration map and bands
quarterly, weights semi-annually or after one cycle, redesign annually or on a pricing / product / segment change.
Track score coverage (GitLab carried >95% of accounts as a company-level yearly goal) and band volatility.
**Publish the migration matrix *before* the rescore lands** and tell each CSM which of their accounts move and why.
A silent rescore destroys trust once, permanently.

**Anti-gaming.** Goodhart's law applies the moment the score is tied to comp, QBR review or CSM performance. The
controls, one per gaming route, are in `references/governance.md` §4 — read it before any comp discussion. The two
that matter most: **mandatory written justification on CSM sentiment, with published per-CSM sentiment-vs-outcome
calibration** (a CSM whose greens churn at 3× the average is miscalibrated, not unlucky), and **overrides capped at
≤10% of portfolio per quarter, each with an expiry**. **The tell that gaming is happening:** score distribution
improves while retention does not — plot them on one axis every quarter.

**Never compensate on the health score itself.** Compensate on retention and expansion outcomes and use the score to
allocate attention; the moment the score is the target it stops being a measurement.

## Output Template

**Brief is the default.** Emit this; offer the rest.

```markdown
**<Company> health score — <verdict>. <the number that proves it>.**

<Two sentences naming the single largest defect or design decision, with its evidence and provenance tag.>

**Do:** <named action> — <owner> by <date>. Expected effect: <what moves, by how much>.
Confidence: <high / medium / low> (<X>/7 families). What would change it: <the falsifier>.

*Full specification, backtest and coverage ledger on request.*
```

**Full** emits the Specification, appends the Backtest Report when outcome data exists (`references/calibration.md`
§8), and emits the Audit block in Audit mode (`references/audit-checklist.md`).
**Copy `assets/score-spec-template.md` verbatim** for Full — never retype it from memory. It carries the Internal
marking and direction line, the Bottom Line header table (predicts · population · archetype · weight method ·
validated? · red threshold · score coverage · as-of · confidence), all nine sections — §1 Prediction Sentence · §2
Dimensions with the hygiene line and rejected candidates · §3 Scoring Functions · §4 Non-Compensatory Layer · §5
Segmentation · §6 Bands and Plays · §7 Explainability Contract · §8 Remediation Roadmap · §9 Governance — and the
Label & History Sufficiency table. Every artifact, Brief or Full, closes with these two blocks:

```markdown
### Coverage Ledger
| Signal family | Source checked | Status | In score? | Weight | Oldest input | Notes |
|---|---|---|---|---|---|---|
| Product usage & adoption | | ✅/⚠️/❌ | | | | |
| Commercial & contract | | | | | | |
| Relationship & engagement | | | | | | |
| Support & reliability | | | | | | |
| Sentiment & VoC | | | | | | |
| Billing & payment | | | | | | |
| Firmographic & external | | | | | | |

**Coverage: X / 7 families (Y%) → confidence capped at <level>** (`R23`). Blind spots: <which families are missing,
and what they typically hide.> Anything absent elsewhere is `UNKNOWN — requires <source/field>`, never a benchmark.

### Assumptions
| # | Assumption | Why it was needed | If wrong |
|---|---|---|---|
| 1 | Label = non-renewal or downsell ≥15% of ARR | No label specified; `cs-context` §6 silent | A logo-only label drops every shrinkage event; ARR capture falls and the usage dimensions lose weight |
| 2 | Horizon 120 days to the opt-out deadline | Notice period present, target horizon not stated | At 90 days AUC rises and the save window shrinks below one motion; at 180 days expect more false reds |
| 3 | Red threshold at the worst decile | No capacity in `coverage-and-capacity` or `cs-context` §3 | If real capacity is half that, the red list is twice what the team can work and CSMs start ignoring all of it |

*(One row per default taken, each with a concrete consequence — "may affect results" is not one. Omit only when nothing was assumed, never silently.)*
```

## Quality Bar

- [ ] The prediction sentence is written, with event, horizon, unit and population all named, the horizon measured to the **opt-out deadline** (`R1`) and labels dated at `decision_date` (`R24`)
- [ ] Every dimension maps to one of the seven signal families, passed the four-axis admission test, and carries an explicit transform, baseline, NA rule and staleness rule; rejected candidates listed with the axis they failed
- [ ] Weight hygiene verified — sums to 100, none <5%, none >35%, count 5–8, ≥50% leading — and the derivation method named with its sample size and limits
- [ ] Missing data can never produce a green: the data-sufficiency floor is stated and enforced, and override caps carry an evidence source and an expiry (`R2`)
- [ ] Segmentation justified by a *relationship* difference, not a level difference; backtest uses time-respecting splits
- [ ] Leakage features named and excluded; accuracy is not a headline metric, base rate is reported first, and capture is reported in **ARR**, not only logos
- [ ] Red threshold derived from stated CSM capacity with the arithmetic shown (`R13`)
- [ ] No probability stated without a fitted calibration map and its backtest cited (`R22`)
- [ ] Explainability contract answers all six questions, reason codes attribute to the delta, and every band has a play with action · owner · date · expected effect · success measure
- [ ] Coverage Ledger over all seven families, plus label & history sufficiency (`R23`)
- [ ] Gaps written `UNKNOWN — requires X` with no benchmark substituted for a company-specific figure, and every default taken appears in the Assumptions table with a concrete consequence
- [ ] Marked Internal; no health score, band or ARR-at-risk figure drafted for a customer (`R18`)
- [ ] The words "will churn", "guaranteed" and "100% accurate" do not appear
- [ ] Brief emitted by default, Full only on request; no competing CS platform named

## Anti-Patterns

| Anti-pattern | Correction |
| --- | --- |
| Weights chosen in a workshop and never revisited | Derive from outcomes (Step 5); until then label them v0 provisional, with a rationale per dimension |
| No stored score history | Fix first. Backtesting, drift detection and velocity are impossible without it, and it cannot be recovered retroactively |
| Reporting accuracy | At a 5% base rate a do-nothing model is 95% accurate. Report base rate, top-decile lift, ARR capture, PR-AUC |
| Random split, or snapshotting T−90 from the renewal date | Split temporally, and snapshot from the opt-out deadline — otherwise you leak the future and score decisions already made |
| 15+ measures, weights nobody remembers | 5–8 dimensions. Contradictory measures cancel and every account lands 60–75 |
| One composite with no sub-scores, or a purely compensatory rollup | Publish the pillars always — a 68 has hundreds of causes needing different plays — and add override caps so fatal conditions cannot average away |
| Missing data scored as green — or as red | NA plus a data-sufficiency floor. An instrumentation failure is an ops defect, not a customer state |
| Score dominated by NPS and ticket counts | ≥50% of weight on leading dimensions; lagging inputs turn red after the decision |
| Raw logins or MAU as the adoption proxy, or a cohort percentile reported as absolute health | Value events against the account's own baseline — a login is not an outcome; and percentiles are zero-sum, so use them for ranking only |
| Thresholds at round numbers | Cut where the observed loss rate changes, then adjust to CSM capacity |
| A dimension nobody can act on | Reject it at the admission test, or move it to segmentation |
| Score tied to CSM comp, or rescored silently | Compensate on retention outcomes and use the score to allocate attention; publish the migration matrix before a rescore and tell each CSM which accounts move and why |
| Ignoring ARR weighting | A model catching 80% of logos and 30% of dollars has failed. Report both |
| "0.87 probability" from an uncalibrated rubric | Bands and deciles until a calibration map is fitted and backtested |
| Asking the user something `cs-context` already answers | Read the file. Ask only what changes the spec — batched, tappable, recommended default first |
| Filling a missing input with a plausible value | Read it, ask it, or mark it `UNKNOWN`. Then register it with its consequence |

## Related Skills

| Skill | Relationship |
| --- | --- |
| `cs-context` | **Run first.** Commercial model, notice period, activation event, sources, existing score's trust level |
| `churn-risk` | Consumes these bands. Its seven-family sweep is the per-account version of what this skill makes persistent |
| `churn-postmortem` | **Feeds back.** Every loss that was Green at T−90 is a defect in this model; its autopsy updates the dimension set |
| `renewal-forecast` | Consumes the calibrated score→probability map. Systematic divergence between score and human forecast means one of the two is broken — the cheapest ongoing validation available |
| `cs-data-audit` | **Runs before** a sparse-data Design — produces the instrumentation roadmap this skill's gap list references |
| `book-of-business-triage` · `expansion-finder` | Consume the bands: the triage queue is built from them and the green-but-falling cell, and every expansion play is gated on the health floor defined here (`R8`) |
| `coverage-and-capacity` | Supplies the capacity number that sets the red threshold in Step 9 |
| `save-play` · `proactive-outreach` | Own every word that reaches a customer. This skill hands them a band, never a draft |

## Going Deeper

| Read | When |
| --- | --- |
| `references/model-a

…(truncated)