# Pm Metrics Critic

> PM metrics critic

- Skill: `kalyvask/pm-metrics-critic` (Agent Skill)
- Install (CLI): `npx skillmds@latest add kalyvask/pm-metrics-critic`
- Raw SKILL.md: https://api.skillmd.com/api/skills/kalyvask/pm-metrics-critic/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Coding & Dev Tools
- Author: kalyvask (https://skillmd.com/u/kalyvask)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/kalyvask/pm-metrics-critic

---


# PM metrics critic

Reviews a set of metrics — a dashboard, a success-criteria section, a North Star proposal — against `decision-making/metrics.md`. The core test: does each metric, if it moved, tell us something *strategic*, or does it just tell us the product is being used? Most metric sets fail this test and the skill's job is to say so concretely.

## When to use

Use this when:
- The user shares a dashboard or KPI list and asks "are these the right metrics?"
- The user is locking success criteria for a launch or a quarterly OKR
- The user is proposing or revisiting a North Star metric
- A metric "moved" and the team is debating whether the underlying assumption was actually validated

Don't use this when:
- The user wants a full PRD critique — use `pm-prd-drafter`
- The user wants to build a metric *tree* from scratch (mapping inputs → outputs → outcomes) — that's a longer exercise; reference `decision-making/metrics.md` § metric trees and walk it manually
- The user is debugging a specific dashboard query or instrumentation bug — that's an engineering task

## How to apply

1. **Pull the strategy assumptions.** Reference `decision-making/metrics.md` § "Match metrics to strategy assumptions." For the work in question, list the load-bearing strategic assumptions. If the user can't articulate them, surface that gap before evaluating any metric — you can't grade metrics without strategy.

2. **For each proposed metric, ask: what assumption would this falsify?** If a metric can't falsify any assumption — if it can only confirm "the product is being used" — it's a vanity metric. Mark it as such.

3. **Run the segment-level decomposition.** Aggregate metrics hide segment failures. If the strategy assumption is segment-specific ("top creators will adopt"), the metric must be segment-specific. Top-line DAU does not validate a top-creator assumption.

   The intensity version of the same check: for any love or satisfaction metric (Sean Ellis, NPS, CSAT), a reported mean has thrown away the data. Ask for the distribution and the band composition — who loves it, who kind of likes it, who isn't interested — what the two engaged bands have in common, and what separates them. "We're at 31% very disappointed" is not actionable; "our 31% are all ops leads who use the shared workspace at 6× the rate of everyone else" is. See `decision-making/metrics.md` § "Sean Ellis test" and `frameworks/03-pre-pmf-validation.md` § "Lovers, likers, and the disinterested."

4. **Run the standalone-vs-counterweight check.** From `decision-making/metrics.md` § counter-metrics: every primary metric should have a counter-metric that catches the obvious gaming path. Engagement up + retention down is a different story than engagement up + retention flat. Flag any metric that ships without a counter.

5. **Run the leading-vs-lagging check.** A success criterion of "ARR by Q4" tells you nothing in weeks 1–8. Identify the leading indicator that would predict the lagging metric *early enough to act on it*.

6. **Recommend a replacement set.** Not "track more metrics." A focused list of 3–5 metrics that each map to a strategic assumption, with a counter-metric for each.

## Output structure

```
## TL;DR
[One-paragraph honest take. Are these metrics measuring the strategy or the activity?]

## Strategic assumptions in play
1. [Assumption 1 — load-bearing for this work]
2. [Assumption 2]
3. ...

## Metric-by-metric
| Metric | What it would falsify | Verdict |
|---|---|---|
| [Metric A] | [Assumption / nothing — vanity] | 🟢 strategic / 🟡 weak / 🔴 vanity |
| [Metric B] | ... | ... |

## Failure modes flagged
- **Aggregate masking segment failure:** [Which metric, what segment is hidden]
- **No counter-metric:** [Which metric, what the obvious gaming path looks like]
- **Lagging-only:** [Which metric, what leading indicator should sit alongside it]
- **Decoupled from strategy:** [Which metric, what assumption it claims to validate but doesn't]

## Recommended replacement set
1. **[Primary metric]** — falsifies *[assumption]*. Segment: [specific]. Threshold: [specific]. Counter-metric: [specific].
2. **[Primary metric 2]** — ...
3. ...

## What an exec will press on
[The 1-2 metrics questions the team will be asked that they're not yet ready for.]
```

## What good looks like

- Every recommended metric is named with a *segment* and a *threshold*. "Adoption" is not a metric. "Adoption among top-decile creators reaching 25% in 60 days" is.
- Each metric has a counter-metric. No exceptions.
- The North Star metric (if one is proposed) is a single metric the entire org could agree to optimize, not a basket of five things.
- Vanity metrics are called out explicitly. "DAU is a vanity metric for this strategy because the assumption is about *which* users adopt, not *how many*" — that level of specificity.
- Leading indicators are named. The team should not have to wait until Q4 to know if Q4 is on track.

## Anti-patterns

- Approving a dashboard because it has "good metrics on it" without checking what they would falsify.
- Recommending more metrics. Metric proliferation is itself a failure mode — most dashboards need fewer, sharper metrics, not more.
- Accepting "engagement" as a metric. Engagement is a category. Engagement-with-what, by-whom, doing-what, is the metric.
- Letting a North Star metric ship without naming the strategic bet it represents. If the team can't say "if North Star moves, it means [specific strategic claim] is true," the North Star isn't load-bearing.
- Ignoring the counter-metric requirement because the team "would never game it." Gaming is rarely intentional; it's structural. The counter-metric exists to catch the structural drift.