alexe-ev Detect Performance Signals
Purpose
Help teams identify, contextualize, and interpret signals of performance change — distinguishing real shifts from noise, and regressions from improvements.
Skill type
Conceptual skill with calculation-aware components
Use this skill when
- A metric has moved unexpectedly (up or down)
- A team is unsure whether a change is meaningful or noise
- An alert has fired and needs to be investigated
- Post-launch monitoring has surfaced an anomaly
Do not use this skill when
- The goal is structured experiment result analysis (use analyze-experiment-results)
- No metric baseline or monitoring exists
Required inputs
- Metric that changed
- Observed change (magnitude and direction)
- Time period of the change
Optional inputs
- Baseline or expected value
- Recent product changes or events
- Segment breakdown
- Data source and methodology
Upstream context
Works best when:
- Metrics framework is defined
- Dashboard is in place
- Historical baseline exists
If upstream context is missing
If no baseline exists, produce a signal investigation framework and flag that without baselines, changes cannot be assessed.
Downstream handoff
Output can feed:
- formulate-experiment-hypothesis (unexplained signal → investigation → experiment)
- analyze-funnel-retention-cohorts (drill into funnel or cohort to diagnose)
- identify-problem-opportunity (signal becomes a product problem to investigate)
Instructions
- Establish the baseline (what was normal before the change).
- Quantify the change: magnitude, direction, duration.
- Check for data quality issues: tracking change, instrumentation error, data lag.
- Identify potential causes: product changes, external events, seasonality, data issues.
- Determine if the change is statistically meaningful vs. noise.
- Recommend investigation path: more data, qualitative research, or experiment.
Output
Provide:
- Signal summary: what changed, by how much, when
- Baseline context
- Data quality assessment
- Potential causes (ordered by likelihood)
- Assessment: noise / meaningful signal / regression / improvement
- Recommended next steps
Risks / caveats
- Avoid premature conclusions — investigate causes before acting
- Check instrumentation before assuming a real product change
- Seasonality and external events are common and frequently overlooked