Data Analysis Standard Skill
Turn raw numbers into product decisions. Structure every analysis with a clear question, methodology, finding, and recommended action.
Analysis Framework: The 4-Question Method
Every analysis starts here:
- What changed? (describe the metric and its movement)
- Why did it change? (root cause — segment, funnel step, cohort, channel)
- So what? (business or product impact)
- Now what? (recommended action with confidence level)
Never deliver data without answering all four. A chart with no narrative is not an analysis.
Metric Triage Template
Use when a metric has moved unexpectedly:
METRIC: [Name]
MOVEMENT: [X% change over Y period]
BASELINE: [What was normal]
SEGMENTATION CHECK:
- By platform (iOS / Android / Web)?
- By user cohort (new / returning / power users)?
- By acquisition channel?
- By geography?
- By plan/tier?
ROOT CAUSE HYPOTHESIS:
1. [Most likely explanation] — Evidence: [data point]
2. [Alternative explanation] — Evidence: [data point]
3. [Ruling out] — Eliminated because: [reason]
CONCLUSION: [Single sentence answer to "why did this change?"]
CONFIDENCE: [High / Medium / Low] — based on [data available]
Funnel Analysis Structure
| Stage |
Metric |
Current |
Benchmark/Target |
Drop-off % |
Notes |
| [Top of funnel] |
[Users] |
[N] |
[N] |
— |
|
| [Step 2] |
[Users] |
[N] |
[N] |
[X%] |
|
| [Step 3] |
[Users] |
[N] |
[N] |
[X%] |
|
| [Conversion] |
[Users] |
[N] |
[N] |
[X%] |
|
Biggest drop-off: [Step X → Step Y] — Hypothesis: [reason]
Recommended investigation: [specific query or test]
Cohort Analysis Guidelines
Always define:
- Cohort definition: [What groups users — signup week, first action, plan type]
- Retention metric: [What counts as retained — login, core action, revenue]
- Retention window: [D1, D7, D30, W4, M3, etc.]
Output a cohort retention table and annotate:
- Baseline retention for each cohort
- Cohorts that over/underperform and why (feature launch? campaign? seasonal?)
- Trend direction across cohorts (improving / declining / stable)
Stakeholder Analysis Output Format
[Analysis Title] — [Date]
Question being answered: [Specific question in plain English]
Time period: [Date range]
Data source: [Where data comes from]
Finding:
[1–2 sentence plain-English summary of what the data shows]
Key chart / table: [Include or describe]
Root cause: [Best explanation with evidence]
Confidence level: [High / Medium / Low] — [reason]
Recommended action:
- [Immediate action — owner, timeline]
- [Investigation needed — what to check next]
- [Monitoring — what metric to watch and at what cadence]
What this analysis does NOT tell us: [Important caveat — what data is missing or what can't be concluded]
Required Inputs
Ask the user for these if not provided:
- Metric or question being investigated
- Time period (what changed, from when to when)
- Data available (which segments, sources, or queries you have access to)
- Business context (what decision this analysis informs)
- Audience (who will read this — exec / team / data team)
Scoring Rubric (0–40)
Score any output of this skill before handing it over; 32+ is ship-quality.
| Dimension |
0 |
5 |
10 |
| Four-question completeness |
Describes what changed and stops |
Covers what/why but "so what / now what" are thin |
All four answered with proportionate depth; the "now what" is decision-ready |
| Evidence behind the root cause |
Root cause asserted from intuition |
One supporting data point, alternatives unexamined |
Root cause tested against at least one rival explanation, with the discriminating evidence shown |
| Uncertainty honesty |
Reads as certain; no confidence statement |
Confidence stated but not justified |
Confidence level justified, and "what the data cannot tell us" names the real blind spots, not token ones |
| Actionability |
Findings with no action |
Action named but ownerless or dateless |
Recommended action has an owner, a timeline, and a stated expected effect worth checking later |
Quality Checks
Anti-Patterns
Guidelines
- Always state what the data cannot tell you — never oversell confidence
- Correlations are not causation — flag this every time
- If the user has no baseline, recommend establishing one before drawing conclusions
- Recommend the simplest chart for each finding: bar for comparison, line for trends, scatter for correlation, table for detailed breakdowns
- Always specify the time window — "conversion dropped" is meaningless without "from X to Y over Z period"
1---2name: data-analysis-standard-23description: Structure a product data analysis, metric deep-dive, funnel analysis, or cohort study. Use when asked to analyse product metrics, investigate a drop in conversion, explain a data change to stakeholders, or find the root cause of a metric movement. Produces a structured analysis with question, root cause, confidence level, and recommended action.4---56# Data Analysis Standard Skill78Turn raw numbers into product decisions. Structure every analysis with a clear question, methodology, finding, and recommended action.910## Analysis Framework: The 4-Question Method1112Every analysis starts here:131. **What changed?** (describe the metric and its movement)142. **Why did it change?** (root cause — segment, funnel step, cohort, channel)153. **So what?** (business or product impact)164. **Now what?** (recommended action with confidence level)1718Never deliver data without answering all four. A chart with no narrative is not an analysis.1920---2122## Metric Triage Template2324Use when a metric has moved unexpectedly:2526```27METRIC: [Name]28MOVEMENT: [X% change over Y period]29BASELINE: [What was normal]3031SEGMENTATION CHECK:32- By platform (iOS / Android / Web)?33- By user cohort (new / returning / power users)?34- By acquisition channel?35- By geography?36- By plan/tier?3738ROOT CAUSE HYPOTHESIS:391. [Most likely explanation] — Evidence: [data point]402. [Alternative explanation] — Evidence: [data point]413. [Ruling out] — Eliminated because: [reason]4243CONCLUSION: [Single sentence answer to "why did this change?"]44CONFIDENCE: [High / Medium / Low] — based on [data available]45```4647---4849## Funnel Analysis Structure5051| Stage | Metric | Current | Benchmark/Target | Drop-off % | Notes |52|---|---|---|---|---|---|53| [Top of funnel] | [Users] | [N] | [N] | — | |54| [Step 2] | [Users] | [N] | [N] | [X%] | |55| [Step 3] | [Users] | [N] | [N] | [X%] | |56| [Conversion] | [Users] | [N] | [N] | [X%] | |5758**Biggest drop-off:** [Step X → Step Y] — Hypothesis: [reason]59**Recommended investigation:** [specific query or test]6061---6263## Cohort Analysis Guidelines6465Always define:66- **Cohort definition:** [What groups users — signup week, first action, plan type]67- **Retention metric:** [What counts as retained — login, core action, revenue]68- **Retention window:** [D1, D7, D30, W4, M3, etc.]6970Output a cohort retention table and annotate:71- Baseline retention for each cohort72- Cohorts that over/underperform and why (feature launch? campaign? seasonal?)73- Trend direction across cohorts (improving / declining / stable)7475---7677## Stakeholder Analysis Output Format7879### [Analysis Title] — [Date]8081**Question being answered:** [Specific question in plain English]82**Time period:** [Date range]83**Data source:** [Where data comes from]8485**Finding:**86> [1–2 sentence plain-English summary of what the data shows]8788**Key chart / table:** [Include or describe]8990**Root cause:** [Best explanation with evidence]9192**Confidence level:** [High / Medium / Low] — [reason]9394**Recommended action:**951. [Immediate action — owner, timeline]962. [Investigation needed — what to check next]973. [Monitoring — what metric to watch and at what cadence]9899**What this analysis does NOT tell us:** [Important caveat — what data is missing or what can't be concluded]100101---102103## Required Inputs104105Ask the user for these if not provided:106- **Metric or question** being investigated107- **Time period** (what changed, from when to when)108- **Data available** (which segments, sources, or queries you have access to)109- **Business context** (what decision this analysis informs)110- **Audience** (who will read this — exec / team / data team)111112## Scoring Rubric (0–40)113114Score any output of this skill before handing it over; 32+ is ship-quality.115116| Dimension | 0 | 5 | 10 |117|---|---|---|---|118| Four-question completeness | Describes what changed and stops | Covers what/why but "so what / now what" are thin | All four answered with proportionate depth; the "now what" is decision-ready |119| Evidence behind the root cause | Root cause asserted from intuition | One supporting data point, alternatives unexamined | Root cause tested against at least one rival explanation, with the discriminating evidence shown |120| Uncertainty honesty | Reads as certain; no confidence statement | Confidence stated but not justified | Confidence level justified, and "what the data cannot tell us" names the real blind spots, not token ones |121| Actionability | Findings with no action | Action named but ownerless or dateless | Recommended action has an owner, a timeline, and a stated expected effect worth checking later |122123## Quality Checks124125- [ ] Analysis answers all 4 questions: what changed, why, so what, now what126- [ ] Root cause has evidence (not just hypothesis)127- [ ] Confidence level is stated and justified128- [ ] What the data cannot tell us is explicitly named129- [ ] Recommended action includes an owner and timeline130131## Anti-Patterns132133- [ ] Do not present correlations as causation — always state the distinction explicitly134- [ ] Do not report a metric movement without stating the time window and comparison baseline135- [ ] Do not skip the "so what" — raw observations without recommended actions are incomplete analysis136- [ ] Do not overstate confidence — label hypotheses clearly and note what data would be needed to confirm them137- [ ] Do not ignore segment breakdowns — aggregate metrics can mask opposing trends in sub-segments138139## Guidelines140141- Always state what the data *cannot* tell you — never oversell confidence142- Correlations are not causation — flag this every time143- If the user has no baseline, recommend establishing one before drawing conclusions144- Recommend the simplest chart for each finding: bar for comparison, line for trends, scatter for correlation, table for detailed breakdowns145- Always specify the time window — "conversion dropped" is meaningless without "from X to Y over Z period"
gabrielmoreira