activation-moment-validator
Agent: Growth Lead
L1 growth leader (1x) responsible for distribution strategy, activation signal definition, retention modelling, growth model design, and growth loop optimisation.
Department ethos: ideal-data-growth.md Tool policy: allowed-tools.yaml
Skill Description
The activation moment validator tests whether the defined activation event actually predicts long-term retention by running correlation and survival analyses between the activation signal and Day 7/30/90 retention cohorts, ensuring the growth team optimizes toward the right behavioural milestone.
When to Use
- When the activation signal has been defined but not yet validated with retention data.
- When retention rates decline despite stable or improving activation rates, suggesting a misaligned definition.
- When a product pivot changes the core value proposition and the existing activation moment may no longer apply.
- When the team debates which user action best represents the "aha moment."
Workflow
- Retrieve activation definition: Pull the current activation signal definition including the event, threshold, and time window (e.g., "completed first project within 48 hours of signup").
- Build cohorts: Segment users into activated vs. non-activated cohorts using the defined signal. Ensure cohorts are large enough for statistical validity (minimum 500 users per group).
- Run retention analysis: Compute Day 7, Day 30, and Day 90 retention rates for each cohort. Calculate the retention lift (activated retention minus non-activated retention).
- Survival analysis: Plot Kaplan-Meier survival curves for both cohorts. Test for significant separation using the log-rank test. A valid activation moment should produce a statistically significant (p < 0.05) separation in survival curves.
- Test alternative signals: If the current signal shows weak predictive power (retention lift < 10 percentage points), test 3-5 alternative candidate actions. Rank by retention lift and statistical significance.
- Deliver validation report: Produce a report with retention cohort analysis, survival curves, p-values, and a validated/invalidated verdict. If invalidated, recommend the strongest alternative signal.
Anti-Patterns
- Validating with small cohorts: Running the analysis on fewer than 500 users per group produces unreliable results. Why: small samples amplify noise and produce confidence intervals too wide to act on.
- Confusing correlation with causation: A high correlation between an action and retention does not prove the action causes retention. Why: power users both activate and retain because of inherent motivation, not because the activation event drove retention.
- One-time validation: Treating validation as a one-time exercise ignores that the relationship between activation and retention evolves as the product and user base change. Why: what predicted retention at 1,000 users may not predict retention at 100,000 users.
Output
Success:
- A validation report containing cohort retention rates, retention lift, survival curves with log-rank test results, and a validated/invalidated verdict with recommended next steps.
Failure:
- The activation signal shows no statistically significant correlation with retention. Report the analysis, the tested alternatives, and recommend the strongest candidate for re-definition.
Related Skills
activation-signal-definer-- defines the activation signal that this skill validates.retention-model-hypothesiser-- the retention model depends on a validated activation moment as its entry point.funnel-analyser-growth-- funnel analysis of the activation flow provides context for why users do or do not reach the activation moment.