Level Churn Analyzer

Level Churn Analyzer

One-liner: Identify game chokepoints by analyzing the last level users attempted before churning, with targeted optimization recommendations.

Core Capabilities

1. Chokepoint Identification - Find levels causing the most user churn
2. Root Cause Analysis - Determine why users churn at specific levels
3. Segment Comparison - Compare level performance across new vs. returning, paid vs. free users
4. Optimization Recommendations - Provide targeted level design and intervention strategies
5. Before/After Comparison - Compare level churn across time periods to measure update/event impact

Quick Start

Example: Standard Level Churn Analysis

User: "Analyze level churn for our Match-3 game"

Skill:

I'll help you analyze level churn. Please provide:

1. **Game Type**: Match-3 ✓
2. **Level Type**: Main / Challenge / Event?
3. **Level Range**: e.g., "1-100" or "all"
4. **User Segment**: All users / New users / Paid users?
5. **Churn Definition**: Days of inactivity (default: 3)
6. **Time Range**: e.g., "Last 30 days" or "2026-04-01 to 2026-05-01"
7. **Project ID**: Your ThinkingData project ID

User:

• Level Type: Main
• Level Range: 1-100
• User Segment: All users
• Churn Definition: 3 days
• Time Range: Last 30 days
• Project ID: 123

Skill: [Executes analysis]

Output:

# Level Churn Diagnostic Report

## Analysis Overview
- Total Challenging Users: 12,500
- Churned Users: 3,240
- Overall 3-Day Churn Rate: 25.9%

## Key Chokepoint Levels

| Rank | Level | Churn % | Severity |
|------|-------|---------|----------|
| 1 | Level 32 | 13.9% | 🔴 P0 |
| 2 | Level 28 | 11.7% | 🟠 P1 |
| 3 | Level 15 | 6.5% | 🟡 P2 |

**Root Cause - Level 32**: Excessive difficulty [High Confidence]
- Pass rate: 18% (below 20% threshold)
- Avg attempts: 4.2x

**Recommended Action**: Reduce obstacle density by 15-20%

Example: Comparison Mode

User: "Compare level churn before and after the update"

Skill:

I'll compare level churn between two periods.

Period A (Before): [Start date] to [End date]
Period B (After): [Start date] to [End date]

Other parameters same as standard analysis.

User: Period A: 2026-04-01 to 2026-04-14, Period B: 2026-04-15 to 2026-04-28

Skill: [Runs parallel analysis and generates comparison report]

Templates and Libraries

This skill uses modular templates for report generation. Reference these files when generating reports:

Report Templates

Calculation Library

Severity Classification with Confidence Intervals

Base Thresholds by Game Genre

Different game types have different "normal" churn baselines. Adjust thresholds accordingly:

Default thresholds (when genre unknown): Use Casual tier.

Severity Classification Table

Sample Size Confidence Guidelines:

Confidence Interval Calculation:

95% CI for churn rate = p ± 1.96 × √(p(1-p)/n)

Where:
- p = observed churn rate (proportion)
- n = sample size (total challenging users)

Display Format with Confidence:

🔴 P0 - Level 32: 13.9% of churn (95% CI: 12.1%-15.7%)
     Level churn rate: 45.2% (95% CI: 42.8%-47.6%)
     Confidence: High (n=2,847 users) ✓

🟠 P1 - Level 28: 11.2% of churn (95% CI: 8.9%-13.5%) ⚠️
     Level churn rate: 38.5% (95% CI: 35.2%-41.8%)
     Confidence: Medium (n=892 users)
     Note: At P0 boundary; consider verification before prioritization

Interaction Flow

Reference: references/interaction_flow.md

This guide provides the complete 4-step interaction workflow:

1. Define Analysis Parameters (7 parameters with validation)

   • Game Type, Level Type, Level Range, User Segment, Churn Definition, Time Range, Project ID
   • Includes input validation templates and production limits

2. Event Confirmation & Data Validation

   • Level event discovery and confirmation
   • Sample size assessment with confidence tiers
   • Data anomaly handling

3. Output Diagnostic Report

   • Report generation using modular templates
   • Standard report ending format

4. Guide Deep Analysis (Optional)

   • Property pre-validation
   • Dimension-based analysis
   • Dynamic report generation

Analysis Methodology

Core Method: Churn Attribution Analysis

Identify "churn-trigger levels" by finding the last level attempted by specific user segments before becoming inactive.

Analysis Target:

• Based on segment selected in Step 2 (All users/New users/Silent users/Paid users/Custom)
• Supports custom segment conditions: registration time, payment status, activity level, current progress
• Analyze which levels users were last at before churning

Key Parameters:

• inactive_days: Days of consecutive inactivity to define churn (default: 3)
• lookback_days: Days to look back for last level (default: 7)
• level_range: Level range to analyze
• user_segment: User segment conditions

Core Metrics:

See references/churn_metrics.md for complete formula reference.

1. Overall Churn Rate (Analysis Overview)

Overall Churn Rate = Churned Users / Total Challenging Users × 100%

Calculation Logic:

1. Churned Users (Numerator): Unique users who churned at any level within analysis time range

   • A user is counted once regardless of how many levels they churned at
   • Deduplicated by user

2. Total Challenging Users (Denominator): Unique users who challenged any level in analysis time range

Example Calculation:

• User A: Churned at Level A → Counted as churned (1 user)
• User B: Churned at Level B → Counted as churned (1 user)
• User C: Did not churn at Level C → Not counted as churned
• User D: Churned at both Level A and B → Counted once (deduplicated)

Result:

• Churned Users: 3 (A + B + D)
• Total Users: 4 (A + B + C + D)
• Overall Churn Rate = 3/4 = 75%

2. Level Churn Rate (Per-Level Metric)

Level Churn Rate = Churns at Level / Total Users Challenging Level × 100%

Calculation Logic:

1. Churn Count (Numerator): Each instance of "last at Level + inactive ≥ threshold" counts as 1 churn

   • Same user can contribute to multiple levels' churn counts (different time periods)
   • Same level same user: Only count the most recent (deduplicate multiple churns at same level)

2. Total Challenging Users (Denominator): Unique users challenging Level in analysis time range

Example Calculation:

• User A:
  • Mar 1: Last at Level A, inactive 3+ days → Level A churn: 1
  • Mar 5: Recalled
  • Mar 10: Last at Level B, inactive 3+ days → Level B churn: 1
  • Mar 15: Challenged Level A again, Mar 20: Last at Level A, inactive → Level A still 1 (overwrites Mar 1)
• User B: Mar 5: Last at Level A, inactive 3+ days → Level A churn: 1
• User C: Challenged Level A but didn't churn → Not counted

Result:

• Level A Churn Count: 2 (User A + User B)
• Level A Total Users: 3 (A + B + C)
• Level A Churn Rate = 2/3 = 66.7%

Level Performance Statistics:

Consistent with churn rate calculation, denominator uses "Total Challenging Users" (deduplicated).

Basic Metrics

Pass/Fail Metrics

User Attribute Metrics

Root Cause Inference Rules

Infer churn causes based on data characteristics. Note: These are heuristic indicators, not definitive causes. Confidence levels indicate diagnostic certainty.

Confidence-Based Inference Guidelines

High Confidence Indicators (Strong signal, act with confidence):

• Multiple high-confidence signals align
• Threshold exceeded by >20% margin
• Consistent across user segments

Medium Confidence Indicators (Probable cause, verify before acting):

• Single medium-confidence signal
• Threshold exceeded by 10-20% margin
• Consider A/B testing interventions

Low Confidence Indicators (Weak signal, investigate further):

• Requires corroborating evidence
• May be normal variation
• Recommend qualitative research (user interviews, session replay)

Compound Diagnosis Format

Connect multiple causes with confidence markers:

Primary: High confidence causes first
Secondary: Medium confidence causes (verify)
Tertiary: Low confidence causes (investigate)

Examples:

Level 12 Analysis:
- Primary: Difficulty mismatch for beginners (High confidence: 78% new users)
- Secondary: Insufficient forgiveness (Medium confidence: 4.2 avg attempts)
- Tertiary: Excessive cooldown (Low confidence: 22h avg retry, borderline)

Inference Limitations Disclaimer

⚠️ Important: Root cause analysis is based on data pattern matching, not causal inference.

• Correlation ≠ Causation: High new user ratio may indicate popular level, not difficulty issue
• Thresholds are heuristics: Adjust based on your game's specific patterns
• Missing variables: External factors (events, updates, competitors) not captured
• Recommend validation: Always combine with user feedback, session replay, or A/B tests

Output Format: Connect multiple causes with "+", and include confidence level: e.g., "Difficulty mismatch [High] + Insufficient forgiveness [Medium]"

Optimization Recommendation Library

Reference: references/recommendations.md

This template provides:

• Universal optimization measures by problem type
• Genre-specific recommendations (8 game types)
• Intervention strategies by timing

Quick Reference

Problem Type → Recommended Actions:

Genre-Specific Quick Links:

• Card: Deck building hints, trial cards
• RPG: Mechanic tutorials, gear drops
• Match-3: Move cap, special items
• Runner: Speed/obstacle adjustment
• SLG: Resource output, strategy hints
• Casual/Sim: Wait time reduction
• MOBA: Newbie matchmaking
• Idle: Offline earnings boost

Comparison Analysis Mode

Compare level churn between two time periods to measure impact of updates, events, or changes.

Trigger Phrases:

• "Compare before and after the update"
• "Analyze the impact of the event"
• "Compare this week vs last week"
• "Did the patch improve level retention?"

Comparison Flow

Step 1 - Identify Comparison Intent

When user mentions comparison keywords, switch to comparison mode:

User: "Compare level churn before and after the update"

Skill: "I'll help you compare level churn between two time periods.

Please provide:
- Period A (Before): Start and end date
- Period B (After): Start and end date
- Other parameters remain the same (game type, level range, user segment, etc.)

Example:
- Period A: 2026-04-01 to 2026-04-14
- Period B: 2026-04-15 to 2026-04-28"

Step 2 - Collect Parameters

Collect standard parameters (game type, level range, user segment, churn definition) plus:

Period A (Baseline):

• Time range: period_a_start to period_a_end
• Label: e.g., "Before Update", "Week 1", "Control Group"

Period B (Comparison):

• Time range: period_b_start to period_b_end
• Label: e.g., "After Update", "Week 2", "Test Group"

Validation Rules:

• Both periods must be valid date ranges
• Periods should not overlap (recommendation)
• Period B should be after Period A (for temporal comparisons)
• Same duration recommended for fair comparison (e.g., both 14 days)

Step 3 - Execute Parallel Analysis

Run standard level churn analysis for both periods with identical parameters, then generate comparison report using references/comparison_report.md.

Deep Analysis Template (Step 4)

After identifying P0/P1 chokepoints in the initial report, proceed with deep analysis.

Reference: Use references/deep_analysis_guide.md for complete guidance including:

• Guidance prompts for user interaction
• Report structure templates (Lean/Core/Full versions)
• Dimension-to-content mapping
• Property discovery guidance
• Analysis dimension selection guide

Quick Reference

Report Depth by Selection:

Dimension Recommendations by Problem Type:

Quick Reference

Important Notes

1. Data Quality: When level challenging user count is low (<100), conclusions may be unreliable. Warn user.
2. Data Latency: Account for data reporting latency, recommend analyzing T-1 or earlier data
3. Event Consistency: Confirm event names match actual project instrumentation
4. Multi-dimensional Comparison: Recommend comparing "New vs. Returning" users - differences are often significant
5. Avoid Over-inference: Root cause analysis is inference-based, mark "Recommend further research validation"

Report Output Requirements

1. Must include: Header overview, Key chokepoint list, Root cause diagnosis, Action recommendations
2. Recommended visualization: Level churn rate trend chart, Chokepoint distribution heatmap
3. Tiered output: Display by P0/P1/P2 priority for operations team processing
4. Actionable: All recommendations must be specific and executable, avoid vague suggestions
5. Must include next-step guidance: Every report must end with clear next options (continue analysis / other levels / end)

Standard Report Ending Format

After every report, use this format to guide users:

---

**📋 Analysis Complete - Choose Next Step:**

**1. Continue Deep Analysis of Current Level**
   → I can analyze any dimension by level event properties
   → Example reply: `Analyze Level {X}, show failure reason distribution`
   → Example reply: `Analyze Level {X}, compare pass rates by power level`
   → Example reply: `Analyze Level {X}, show item usage and completion correlation`

**2. Analyze Other Chokepoint Levels**
   → Reply: `Analyze Level {Y}`

**3. End Analysis**
   → Reply: `End`

---