samarv The Sean Ellis Test (or "The 40% Test") provides a leading indicator of product-market fit. While retention cohorts are the ultimate "lagging" proof of fit, this survey allows you to identify if you have a "must-have" product day one, without waiting months for data to mature.
1. Segment the Audience
Do not survey everyone. Surveying people who have only seen a demo or just signed up will result in "noise."
- Target: Users who have experienced the "core" of the product.
- Criteria: Must have used the product at least twice.
- Recency: Must have used the product within the last 1–2 weeks (active users, not churned users).
- Sample Size: Aim for at least 30–50 responses for valid initial signal.
2. Deploy the Primary Question
Ask the single most important question to gauge "must-have" status:
"How would you feel if you could no longer use [Product Name]?"
- Very disappointed (The "Must-Have" group)
- Somewhat disappointed (The "Nice-to-Have" group)
- Not disappointed
- N/A – I no longer use the product
3. Analyze the Threshold
- 40% or higher "Very Disappointed": You have found a "must-have" vein. You are ready to focus on activation and sustainable growth.
- Below 40%: You are likely in a "commodity" state or targeting the wrong segment. Do not scale marketing yet; you will waste capital.
4. Extract the "Must-Have" Benefit
For the users who said they would be Very Disappointed, run a follow-up qualitative analysis to understand why they care.
Step A: Open-Ended Discovery
Ask: "What is the primary benefit that you get from [Product]?" and "Why is that benefit important to you?"
- Look for recurring language or "hooks" (e.g., "I'm drowning in email").
Step B: Benefit Validation (Multiple Choice)
Once you have 4–5 recurring themes, survey a different group of users.
- Question: "Which of these is the primary benefit you receive?" (Force a choice between the 4–5 themes).
- Question: "What would you use if this product were no longer available?" (Identifies the true competition).
5. Move the Score (The "Lookout" Strategy)
If your score is low (e.g., 10-15%), use this workflow to reach the 40% threshold:
- Isolate the "Very Disappointed" group: Even if it's only 7%, find out who they are and what feature they use.
- Reposition the Hook: Update marketing and landing pages to focus exclusively on the specific benefit that 7% loved.
- Streamline Onboarding: Remove any steps that don't lead directly to that specific benefit. Ensure "Speed to Value" (Aha Moment) happens in the first session.
- Ignore the "Somewhat Disappointed" users: Do not build features for them yet. Trying to please everyone dilutes the product for your "must-have" core.
Example 1: Mobile Security (Lookout)
- Context: A mobile app with backup, phone-finding, and antivirus features had only a 7% PMF score.
- Insight: The 7% who loved it cared exclusively about "Antivirus," even though phone viruses were rare at the time.
- Application: The team repositioned all marketing on "Antivirus" and changed onboarding so the first thing a user saw was an "Antivirus Scanning" progress bar.
- Output: The score jumped to 40% in two weeks because the product now attracted people seeking that specific value and delivered it immediately.
Example 2: Productivity Tool (Xobni)
- Context: Testing the primary benefit of an email plugin.
- Input: Users were asked "Why is finding things faster important to you?"
- Application: Qualitative responses repeatedly used the phrase "I'm drowning in email."
- Output: The marketing was changed to lead with the "Drowning in email?" hook, which significantly increased the conversion of users who eventually became "Very Disappointed" must-have users.
Common Pitfalls
- Surveying Demos: Asking "would you use this?" is useless. Only survey users who have actually performed the core action.
- The "Somewhat Disappointed" Trap: Trying to convert "Somewhat Disappointed" users by adding their requested features. This often results in a mediocre product that is "good for everyone but great for no one."
- Ignoring Context: Not asking why a benefit is important. The "why" provides the emotional context needed for high-performing acquisition copy.
- Premature Scaling: Stepping on the gas (paid ads) when the score is at 15%. This results in high churn and "leaky bucket" syndrome.