Developer Pain Finder
Given a keyword or problem space, scrape real developer complaints from Reddit, Hacker News, dev.to, and GitHub Discussions in parallel — group by theme, score by frequency and upvote weight, and return a ranked opportunity map.
Pre-flight check
tinyfish --version
tinyfish auth status
If not installed: npm install -g tinyfish
If not authenticated: tinyfish auth login
Step 1 — Clarify input
You need:
- Keyword or problem space — e.g. "authentication", "deployment pipelines", "ORM", "Kubernetes", "state management"
If the user hasn't provided one, ask before proceeding. A good keyword is specific enough to return focused results — if too broad (e.g. "JavaScript"), ask them to narrow it down.
Step 2 — Parallel scraping
Fire all 5 agents simultaneously. Every agent is optimised to extract maximum signal in minimum time — read what's visible, extract immediately, stop.
# ── AGENT 1 — Reddit (stealth, search results only) ──────────
tinyfish agent run \
--url "https://www.reddit.com/search/?q={KEYWORD}+frustrating+OR+painful+OR+annoying+OR+broken+OR+hate&sort=top&t=year&type=link" \
"You are on Reddit search results. Your only job is to extract pain points fast.
Read the post titles and score/comment counts visible on this page. Nothing else.
Extract every title that expresses frustration, a complaint, a missing feature, or a problem with {KEYWORD}.
For each one note: the title text, upvote score (if visible), and subreddit.
STRICT RULES — these are non-negotiable:
- Do NOT click any post. Ever. Read titles and snippets only.
- Do NOT scroll. Read only what loads on first render.
- Do NOT navigate anywhere. Stay on this exact page.
- Ignore posts that are not complaints or pain points.
- Stop after reading 20 results maximum.
Return JSON array: [{title, score, subreddit, snippet}]" \
--sync --browser-profile stealth > /tmp/dpf_reddit.json &
# ── AGENT 2 — Reddit r/ExperiencedDevs + topic subreddits ────
tinyfish agent run \
--url "https://www.reddit.com/search/?q={KEYWORD}&restrict_sr=0&sort=top&t=year&type=link&subreddit=webdev+programming+devops+ExperiencedDevs+softwaredevelopment" \
"You are on Reddit search results filtered to developer subreddits. Extract pain points fast.
Read only post titles and visible snippets. Do not interact with the page.
Extract titles expressing: bugs, missing features, poor DX, complexity complaints, tool failures, workflow friction about {KEYWORD}.
For each: title text, subreddit, upvote score if visible.
STRICT RULES:
- Do NOT click anything
- Do NOT scroll
- Read exactly what is on screen, then stop
- Maximum 20 results
Return JSON array: [{title, score, subreddit, snippet}]" \
--sync --browser-profile stealth > /tmp/dpf_reddit2.json &
# ── AGENT 3 — Hacker News ─────────────────────────────────────
tinyfish agent run \
--url "https://hn.algolia.com/?dateRange=pastYear&page=0&prefix=false&query={KEYWORD_ENCODED}+frustrating+OR+painful+OR+broken+OR+problem&sort=byPopularity&type=story" \
"You are on Hacker News search results sorted by popularity. Extract pain points fast.
Read the story titles, point scores, and comment counts visible on this page.
Extract every story title that describes a problem, frustration, complaint, or missing capability related to {KEYWORD}.
For each: title text, points, comment count.
STRICT RULES:
- Do NOT click any story link
- Do NOT paginate
- Do NOT scroll past the first visible set of results
- Read only titles and metadata shown in the listing
- Maximum 15 results
- Stop immediately after extracting — do not explore further
Return JSON array: [{title, points, comments}]" \
--sync > /tmp/dpf_hn.json &
# ── AGENT 4 — dev.to ──────────────────────────────────────────
tinyfish agent run \
--url "https://dev.to/search?q={KEYWORD_ENCODED}+problem+OR+frustration+OR+painful+OR+hate+OR+annoying" \
"You are on dev.to search results. Extract developer pain points fast.
Read only the article titles, reaction counts, and tag labels visible in the search listing.
Extract articles whose titles describe a problem, complaint, frustration, or missing solution related to {KEYWORD}.
For each: title, reaction count if visible, tags.
STRICT RULES:
- Do NOT click any article
- Do NOT scroll more than once
- Do NOT navigate away
- Read only what is visible in the search result cards
- Maximum 15 results then stop immediately
Return JSON array: [{title, reactions, tags}]" \
--sync > /tmp/dpf_devto.json &
# ── AGENT 5 — GitHub Discussions ──────────────────────────────
tinyfish agent run \
--url "https://github.com/search?q={KEYWORD_ENCODED}+pain+OR+frustrating+OR+broken+OR+missing+OR+annoying&type=discussions&s=reactions&o=desc" \
"You are on GitHub Discussions search results sorted by most reactions. Extract pain points fast.
Read only the discussion titles, reaction counts, and repository names visible in the listing.
Extract discussions that describe a real problem, missing feature, DX complaint, or frustration with {KEYWORD}.
For each: title, reaction count, repo name.
STRICT RULES:
- Do NOT click any discussion link
- Do NOT paginate
- Do NOT scroll
- Read only what is visible on first load
- Maximum 15 results then stop
Return JSON array: [{title, reactions, repo}]" \
--sync > /tmp/dpf_github.json &
# ── WAIT ──────────────────────────────────────────────────────
wait
echo "=== REDDIT 1 ===" && cat /tmp/dpf_reddit.json
echo "=== REDDIT 2 ===" && cat /tmp/dpf_reddit2.json
echo "=== HN ===" && cat /tmp/dpf_hn.json
echo "=== DEVTO ===" && cat /tmp/dpf_devto.json
echo "=== GITHUB ===" && cat /tmp/dpf_github.json
Before running, replace:
{KEYWORD} — the topic e.g. Kubernetes, authentication, state management
{KEYWORD_ENCODED} — URL-encoded version e.g. state%20management
Step 3 — Theme grouping and scoring
From all results combined:
1. Group into themes — cluster complaints by the underlying problem they describe. Examples:
- "Poor documentation" — all titles complaining about docs, examples, guides
- "Complex setup / configuration" — anything about setup friction, boilerplate, config hell
- "Performance issues" — slow, memory leaks, timeouts
- "Missing features" — requests for things that don't exist
- "Breaking changes / stability" — complaints about upgrades, deprecations, semver
- "Poor error messages / debugging" — hard to debug, cryptic errors
- "Vendor lock-in / pricing" — cost complaints, switching costs
Don't force complaints into themes — let themes emerge from the data.
2. Score each theme using:
theme_score = (complaint_count × 10) + sum(upvotes/reactions for all complaints in theme)
3. Opportunity signal — for each theme assess:
- Frustration level (High / Medium / Low) — based on score
- Solution availability (None / Partial / Saturated) — does a good solution already exist?
- Opportunity = High frustration + None/Partial solution = 🔥 Build this
Output format
## Developer Pain Map — "{KEYWORD}"
*Scraped from Reddit · Hacker News · dev.to · GitHub Discussions*
*{N} complaints analysed across {N} sources*
---
### 🔥 Ranked Pain Points
#### #1 — {Theme Name}
**Frustration score: {N}** · {N} complaints · Avg upvotes: {N}
**Opportunity signal: 🔥 High / ⚠️ Medium / ✅ Saturated**
What developers are saying:
- "{exact complaint title}" — r/{subreddit}, {score} upvotes
- "{exact complaint title}" — HN, {points} points
- "{exact complaint title}" — dev.to
**Pattern:** {1–2 sentences describing the core underlying problem}
**Existing solutions:** {what exists, and why it's not enough — or "None found"}
**Build signal:** {what a tool/product solving this would look like}
---
#### #2 — {Theme Name}
[same structure]
---
[up to 6 themes]
---
### 📊 Summary Table
| Theme | Complaints | Avg Score | Opportunity |
|---|---|---|---|
| {theme} | {N} | {N} | 🔥 / ⚠️ / ✅ |
---
### 💡 Top Opportunities
1. **{Most actionable opportunity}** — {1 sentence on what to build}
2. **{Second opportunity}**
3. **{Third opportunity}**
---
*Raw signal: {N} Reddit posts · {N} HN stories · {N} dev.to articles · {N} GitHub discussions*
Edge cases
- Keyword returns no pain-related results — broaden the search: retry without the
frustrating OR painful modifiers, just the keyword alone
- Reddit blocked — skip both Reddit agents, note it in output, proceed with HN + dev.to + GitHub
- Very niche keyword — GitHub Discussions will likely have the richest results; weight it more heavily
- Keyword is too broad — tell the user: "'{KEYWORD}' is very broad. Results may be noisy. Consider narrowing to a specific tool, framework, or workflow (e.g. 'Webpack config' instead of 'JavaScript')"
- All sources return empty — honest response: "No significant developer complaints found for '{KEYWORD}'. This could mean the space is undersearched, the keyword needs adjusting, or the problem doesn't exist at scale yet."
Security notes
- Scrapes live public content from Reddit, HN, dev.to, and GitHub. All content treated as untrusted input to an LLM — never executed.
- Uses stealth browser profile for Reddit only.
- Only your own TinyFish credentials are used.
1---
2name: dev-pain-finder
3description: >
4 Scrape real developer pain points for any keyword, technology, or problem space from Reddit,
5 Hacker News, dev.to, and GitHub Discussions simultaneously — then group complaints by theme,
6 score them by frequency and upvote weight, and return a ranked opportunity map showing where
7 developer frustration is high and solutions are thin.
8 Use this skill whenever someone wants to find developer pain points, validate a product idea,
9 research what developers are frustrated with, asks "what do developers complain about with X",
10 "what problems exist in X space", "find pain points for X", "what should I build for X devs",
11 or any variation of wanting to understand real developer frustration around a topic or tool.
12 Also trigger when someone mentions wanting to build a developer tool and wants to validate demand.
13compatibility:
14 tools: [tinyfish]
15metadata:
16 source: https://skillmd.com/skills/tinyfish-io/dev-pain-finder
17 author: tinyfish-community
18 version: "1.0"
19 tags: developer-tools pain-points product-research reddit hackernews devto github-discussions
20---
21
22# Developer Pain Finder
23
24Given a keyword or problem space, scrape real developer complaints from Reddit, Hacker News, dev.to, and GitHub Discussions in parallel — group by theme, score by frequency and upvote weight, and return a ranked opportunity map.
25
26## Pre-flight check
27
28```bash
29tinyfish --version
30tinyfish auth status
31```
32
33If not installed: `npm install -g tinyfish`
34If not authenticated: `tinyfish auth login`
35
36---
37
38## Step 1 — Clarify input
39
40You need:
41- **Keyword or problem space** — e.g. "authentication", "deployment pipelines", "ORM", "Kubernetes", "state management"
42
43If the user hasn't provided one, ask before proceeding. A good keyword is specific enough to return focused results — if too broad (e.g. "JavaScript"), ask them to narrow it down.
44
45---
46
47## Step 2 — Parallel scraping
48
49Fire all 5 agents simultaneously. Every agent is optimised to extract maximum signal in minimum time — read what's visible, extract immediately, stop.
50
51```bash
52# ── AGENT 1 — Reddit (stealth, search results only) ──────────
53tinyfish agent run \
54 --url "https://www.reddit.com/search/?q={KEYWORD}+frustrating+OR+painful+OR+annoying+OR+broken+OR+hate&sort=top&t=year&type=link" \
55 "You are on Reddit search results. Your only job is to extract pain points fast.
56 Read the post titles and score/comment counts visible on this page. Nothing else.
57 Extract every title that expresses frustration, a complaint, a missing feature, or a problem with {KEYWORD}.
58 For each one note: the title text, upvote score (if visible), and subreddit.
59 STRICT RULES — these are non-negotiable:
60 - Do NOT click any post. Ever. Read titles and snippets only.
61 - Do NOT scroll. Read only what loads on first render.
62 - Do NOT navigate anywhere. Stay on this exact page.
63 - Ignore posts that are not complaints or pain points.
64 - Stop after reading 20 results maximum.
65 Return JSON array: [{title, score, subreddit, snippet}]" \
66 --sync --browser-profile stealth > /tmp/dpf_reddit.json &
67
68# ── AGENT 2 — Reddit r/ExperiencedDevs + topic subreddits ────
69tinyfish agent run \
70 --url "https://www.reddit.com/search/?q={KEYWORD}&restrict_sr=0&sort=top&t=year&type=link&subreddit=webdev+programming+devops+ExperiencedDevs+softwaredevelopment" \
71 "You are on Reddit search results filtered to developer subreddits. Extract pain points fast.
72 Read only post titles and visible snippets. Do not interact with the page.
73 Extract titles expressing: bugs, missing features, poor DX, complexity complaints, tool failures, workflow friction about {KEYWORD}.
74 For each: title text, subreddit, upvote score if visible.
75 STRICT RULES:
76 - Do NOT click anything
77 - Do NOT scroll
78 - Read exactly what is on screen, then stop
79 - Maximum 20 results
80 Return JSON array: [{title, score, subreddit, snippet}]" \
81 --sync --browser-profile stealth > /tmp/dpf_reddit2.json &
82
83# ── AGENT 3 — Hacker News ─────────────────────────────────────
84tinyfish agent run \
85 --url "https://hn.algolia.com/?dateRange=pastYear&page=0&prefix=false&query={KEYWORD_ENCODED}+frustrating+OR+painful+OR+broken+OR+problem&sort=byPopularity&type=story" \
86 "You are on Hacker News search results sorted by popularity. Extract pain points fast.
87 Read the story titles, point scores, and comment counts visible on this page.
88 Extract every story title that describes a problem, frustration, complaint, or missing capability related to {KEYWORD}.
89 For each: title text, points, comment count.
90 STRICT RULES:
91 - Do NOT click any story link
92 - Do NOT paginate
93 - Do NOT scroll past the first visible set of results
94 - Read only titles and metadata shown in the listing
95 - Maximum 15 results
96 - Stop immediately after extracting — do not explore further
97 Return JSON array: [{title, points, comments}]" \
98 --sync > /tmp/dpf_hn.json &
99
100# ── AGENT 4 — dev.to ──────────────────────────────────────────
101tinyfish agent run \
102 --url "https://dev.to/search?q={KEYWORD_ENCODED}+problem+OR+frustration+OR+painful+OR+hate+OR+annoying" \
103 "You are on dev.to search results. Extract developer pain points fast.
104 Read only the article titles, reaction counts, and tag labels visible in the search listing.
105 Extract articles whose titles describe a problem, complaint, frustration, or missing solution related to {KEYWORD}.
106 For each: title, reaction count if visible, tags.
107 STRICT RULES:
108 - Do NOT click any article
109 - Do NOT scroll more than once
110 - Do NOT navigate away
111 - Read only what is visible in the search result cards
112 - Maximum 15 results then stop immediately
113 Return JSON array: [{title, reactions, tags}]" \
114 --sync > /tmp/dpf_devto.json &
115
116# ── AGENT 5 — GitHub Discussions ──────────────────────────────
117tinyfish agent run \
118 --url "https://github.com/search?q={KEYWORD_ENCODED}+pain+OR+frustrating+OR+broken+OR+missing+OR+annoying&type=discussions&s=reactions&o=desc" \
119 "You are on GitHub Discussions search results sorted by most reactions. Extract pain points fast.
120 Read only the discussion titles, reaction counts, and repository names visible in the listing.
121 Extract discussions that describe a real problem, missing feature, DX complaint, or frustration with {KEYWORD}.
122 For each: title, reaction count, repo name.
123 STRICT RULES:
124 - Do NOT click any discussion link
125 - Do NOT paginate
126 - Do NOT scroll
127 - Read only what is visible on first load
128 - Maximum 15 results then stop
129 Return JSON array: [{title, reactions, repo}]" \
130 --sync > /tmp/dpf_github.json &
131
132# ── WAIT ──────────────────────────────────────────────────────
133wait
134
135echo "=== REDDIT 1 ===" && cat /tmp/dpf_reddit.json
136echo "=== REDDIT 2 ===" && cat /tmp/dpf_reddit2.json
137echo "=== HN ===" && cat /tmp/dpf_hn.json
138echo "=== DEVTO ===" && cat /tmp/dpf_devto.json
139echo "=== GITHUB ===" && cat /tmp/dpf_github.json
140```
141
142**Before running**, replace:
143- `{KEYWORD}` — the topic e.g. `Kubernetes`, `authentication`, `state management`
144- `{KEYWORD_ENCODED}` — URL-encoded version e.g. `state%20management`
145
146---
147
148## Step 3 — Theme grouping and scoring
149
150From all results combined:
151
152**1. Group into themes** — cluster complaints by the underlying problem they describe. Examples:
153- "Poor documentation" — all titles complaining about docs, examples, guides
154- "Complex setup / configuration" — anything about setup friction, boilerplate, config hell
155- "Performance issues" — slow, memory leaks, timeouts
156- "Missing features" — requests for things that don't exist
157- "Breaking changes / stability" — complaints about upgrades, deprecations, semver
158- "Poor error messages / debugging" — hard to debug, cryptic errors
159- "Vendor lock-in / pricing" — cost complaints, switching costs
160
161Don't force complaints into themes — let themes emerge from the data.
162
163**2. Score each theme** using:
164```
165theme_score = (complaint_count × 10) + sum(upvotes/reactions for all complaints in theme)
166```
167
168**3. Opportunity signal** — for each theme assess:
169- **Frustration level** (High / Medium / Low) — based on score
170- **Solution availability** (None / Partial / Saturated) — does a good solution already exist?
171- **Opportunity** = High frustration + None/Partial solution = 🔥 Build this
172
173---
174
175## Output format
176
177```
178## Developer Pain Map — "{KEYWORD}"
179*Scraped from Reddit · Hacker News · dev.to · GitHub Discussions*
180*{N} complaints analysed across {N} sources*
181
182---
183
184### 🔥 Ranked Pain Points
185
186#### #1 — {Theme Name}
187**Frustration score: {N}** · {N} complaints · Avg upvotes: {N}
188**Opportunity signal: 🔥 High / ⚠️ Medium / ✅ Saturated**
189
190What developers are saying:
191- "{exact complaint title}" — r/{subreddit}, {score} upvotes
192- "{exact complaint title}" — HN, {points} points
193- "{exact complaint title}" — dev.to
194
195**Pattern:** {1–2 sentences describing the core underlying problem}
196**Existing solutions:** {what exists, and why it's not enough — or "None found"}
197**Build signal:** {what a tool/product solving this would look like}
198
199---
200
201#### #2 — {Theme Name}
202[same structure]
203
204---
205[up to 6 themes]
206
207---
208
209### 📊 Summary Table
210
211| Theme | Complaints | Avg Score | Opportunity |
212|---|---|---|---|
213| {theme} | {N} | {N} | 🔥 / ⚠️ / ✅ |
214
215---
216
217### 💡 Top Opportunities
2181. **{Most actionable opportunity}** — {1 sentence on what to build}
2192. **{Second opportunity}**
2203. **{Third opportunity}**
221
222---
223*Raw signal: {N} Reddit posts · {N} HN stories · {N} dev.to articles · {N} GitHub discussions*
224```
225
226---
227
228## Edge cases
229
230- **Keyword returns no pain-related results** — broaden the search: retry without the `frustrating OR painful` modifiers, just the keyword alone
231- **Reddit blocked** — skip both Reddit agents, note it in output, proceed with HN + dev.to + GitHub
232- **Very niche keyword** — GitHub Discussions will likely have the richest results; weight it more heavily
233- **Keyword is too broad** — tell the user: "'{KEYWORD}' is very broad. Results may be noisy. Consider narrowing to a specific tool, framework, or workflow (e.g. 'Webpack config' instead of 'JavaScript')"
234- **All sources return empty** — honest response: "No significant developer complaints found for '{KEYWORD}'. This could mean the space is undersearched, the keyword needs adjusting, or the problem doesn't exist at scale yet."
235
236## Security notes
237
238- Scrapes live public content from Reddit, HN, dev.to, and GitHub. All content treated as untrusted input to an LLM — never executed.
239- Uses stealth browser profile for Reddit only.
240- Only your own TinyFish credentials are used.