ComeOnOliver Feedback Analysis
Overview
Collect user feedback from multiple channels, categorize it, extract patterns, and turn it into prioritized product decisions. Build a systematic process from raw input to actionable insight.
Instructions
Multi-channel collection
PROACTIVE (you ask)
├── In-app surveys (NPS, CSAT, CES)
├── Email campaigns (post-purchase, post-onboarding)
├── User interviews (1:1, 30-60 min)
└── Beta feedback forms
REACTIVE (they tell you)
├── Support tickets (Zendesk, Intercom, Freshdesk)
├── App store reviews (iOS, Android)
├── Social media mentions (Twitter, Reddit, HN)
└── Community forums (Discord, Slack, GitHub Issues)
BEHAVIORAL (they show you)
├── Session recordings (Hotjar, FullStory)
├── Feature usage analytics
└── Drop-off funnels and search queries
Feedback categorization
Classify every piece of feedback on three dimensions:
1. TYPE: Bug report | Feature request | Usability issue | Performance | Praise | Question
2. AREA: Onboarding | Core workflow | Billing | Integrations | Mobile | Account
3. SEVERITY: Critical (blocking) | High (workaround exists) | Medium | Low (nice-to-have)
For high-volume feedback (1000+/month), automate classification:
# feedback_classifier.py — LLM-based batch classification
CLASSIFICATION_PROMPT = """Classify this user feedback:
Feedback: "{text}"
Return JSON:
{{"type": "bug|feature_request|usability|performance|praise|question",
"area": "onboarding|core_workflow|billing|integrations|mobile|account",
"severity": "critical|high|medium|low",
"sentiment": "positive|neutral|negative",
"key_theme": "one phrase summarizing the core issue",
"actionable": true/false}}"""
def classify_feedback(text: str) -> dict:
"""Classify a single feedback item into structured categories."""
response = client.chat.completions.create(
model="gpt-4o-mini",
messages=[{"role": "user", "content": CLASSIFICATION_PROMPT.format(text=text)}],
response_format={"type": "json_object"}
)
return json.loads(response.choices[0].message.content)
Sentiment analysis
NPS (Net Promoter Score): "How likely to recommend? (0-10)". Detractors 0-6, Passives 7-8, Promoters 9-10. NPS = %Promoters - %Detractors. Benchmarks: SaaS +30 good/+50 excellent. The follow-up "Why?" is where insights live.
CSAT: "How satisfied with [interaction]?" (1-5). Measured after specific touchpoints. Target: >80%.
CES (Customer Effort Score): "How easy was it to [task]?" (1-7). Best predictor of churn for specific interactions. Users scoring 1-3 are 4x more likely to churn.
Theme extraction
- Aggregate all feedback from past 30 days
- Sample if high volume (200+ items)
- Tag each item with consistent theme labels
- Rank themes by frequency
- Weight: frequency × severity × segment value
- Cross-reference with behavioral data
Theme report format:
## Feedback Theme Report — [Month Year]
### Top Themes (by weighted frequency)
| # | Theme | Count | Severity | Top Segment | Sample Quote |
|---|-------|-------|----------|-------------|--------------|
| 1 | Slow search | 67 | High | Power users | "Search takes 5+ sec on large projects" |
| 2 | Missing CSV export | 52 | Medium | Enterprise | "Need to get data into BI tools" |
### Churn-Correlated Themes
1. Slow search (34% of churners mentioned)
2. Missing integrations (28%)
Feature request prioritization
RICE Framework:
RICE Score = (Reach × Impact × Confidence) / Effort
Reach: Users affected next quarter (number)
Impact: 3=massive, 2=high, 1=medium, 0.5=low, 0.25=minimal
Confidence: 100%=high, 80%=medium, 50%=low
Effort: Person-months to build
Example — CSV Export:
Reach: 500, Impact: 2, Confidence: 90%, Effort: 1 month
RICE = (500 × 2 × 0.9) / 1 = 900
Kano Model: Classify features as Must-have (expected), Performance (more is better), Delighter (unexpected joy), Indifferent, or Reverse (unwanted).
Feedback-to-action pipeline
Raw Feedback → Classify → Tag Themes → Quantify → Prioritize → Build → Close Loop
The "Close Loop" step is critical: tell users you heard them, tell them when you ship it. This turns frustrated users into advocates.
Examples
Analyze app store reviews for product insights
We have 2,400 app store reviews (iOS + Android) from the past 6 months. Export them and analyze for recurring themes, sentiment trends over time, and feature requests. Identify the top 5 issues driving negative reviews and recommend specific product changes. Include a breakdown by star rating and platform.
Build a feedback collection system
Our B2B SaaS product has 3,000 active users across 200 companies. We currently collect feedback only through support tickets. Design a multi-channel feedback system — in-app surveys, NPS, feature request portal, and automated review monitoring. Include the timing triggers, question wording, and how to aggregate insights into a monthly report.
Prioritize a feature backlog using user feedback
We have 45 feature requests from the past quarter across support tickets, sales calls, and NPS comments. Score each using RICE framework, cross-reference with churn data, and produce a prioritized backlog with the top 10 features to build next quarter. Include the data sources and confidence level for each score.
Guidelines
- Always close the feedback loop — tell users when their requested feature ships
- Use CES (Customer Effort Score) for specific interaction feedback; it's the best churn predictor
- Never rely on a single feedback channel; aggregate from proactive, reactive, and behavioral sources
- Weight feedback by user segment value, not just raw frequency
- Cross-reference qualitative themes with quantitative behavioral data to validate patterns
- Automate classification for high-volume feedback (1000+/month) using LLM batch processing
- Run theme reports monthly and share with product, engineering, and support teams