# Market Intel Research

> Industry RESEARCH + market intelligence. Use when the user says "research <vertical>", "study the market", "industry brief", "spar <vertical>", "what changed in the pipeline", "fresh seams", or wants a market brief and a spar report against the last locked pipeline snapshot. Runs the market-research engine to produce actionable niche intelligence from public data.

- Skill: `amirulskulker/market-intel-research` (Agent Skill)
- Install (CLI): `npx skillmds@latest add amirulskulker/market-intel-research`
- Raw SKILL.md: https://api.skillmd.com/api/skills/amirulskulker/market-intel-research/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: DevOps & Infra
- Author: amirulskulker (https://skillmd.com/u/amirulskulker)
- Updated: 2026-09-22
- Page: https://skillmd.com/skills/amirulskulker/market-intel-research

---


# Market Intel — Research Skill

Turn a vague "study this market" into a ranked, evidence-backed, actionable
brief — and challenge it against your own last snapshot.

## Ground rules (non-negotiable)

1. **Public data only.** Everything comes from legally scraped public
   sources (OSM, public directories, search). No login, no CAPTCHA bypass,
   no ToS violations. If a source requires credentials, stop and say so —
   don't improvise scraping.
2. **Evidence over vibes.** Every claim in a report carries a source (which
   crawl, which report, which page). No source = mark it as estimate.
3. **Revenue/scale are PROXIES.** Private micro-business revenue is not
   public. Use review counts × category benchmarks → band + confidence.
   Never fake precision. Say "estimated RM X–Y", not "RM X".
4. **Free-first stack.** Prefer $0 tools (OSM, browser automation) over paid
   scrapers. Only propose paid tools when the free path is genuinely blocked.
5. **Respect the environment:** the engine is Python; write temp scripts to
   a scratch dir rather than inline shell one-liners.

## The pipeline (research → plan → spar)

### Stage 1 — Crawl (who exists)
```
osm_crawl --area <area> [--category <vertical>]      # baseline POI density
gmaps_enrich --query "<vertical> <area>" --limit N   # phone/rating/reviews/website
```
- Free, keyless baselining first; enrichment second.
- Enrichment is fragile (layout changes): early-exit on 2 no-new-result
  cycles; incremental saves so timeouts don't lose data.

### Stage 2 — Report (what the data says)
```
niche_report --area <area>          # per-niche opportunity scores 0-100
pitch_list --area <area>            # reachable businesses with NO website, ranked
```
- The **grey line** is the wedge: businesses that are active (reviews) but
  digitally invisible (no website). That gap = the sales opportunity.

### Stage 3 — Suggest (the plan)
```
suggest --area <area> --tier flash|pro|ultra
```
- LLM stage ONLY where judgment is needed. Reads the niche report + pitch
  list → produces: niche ranking with willingness-to-pay, TOP N targets,
  pitch angles, wedges, risks, next steps.
- Terse, decisive output: "decisions and reasons", not hedging.

### Stage 4 — Spar (challenge your own plan)
```
spar <vertical> --against <snapshot>
```
- Re-run against the last locked pipeline snapshot. Output: what changed,
  what's now uncontested, what's saturated, and a verdict (action/reject/
  defer).

### Stage 5 — Persist + query
```
pgvector_load --area <area>         # push to vector store for semantic queries
```
- Idempotent (dedupe by name+coords). Keeps its own table — never mixes
  into other tenants' corpora.

## Output discipline

- **Brief:** `STUDY_<vertical>_<area>_brief.md` — problem, data, ranking,
  targets, wedge, next action.
- **Spar:** `STUDY_<vertical>_spar.md` — diff against snapshot + verdict.
- Both: recommendations are ranked by a score (0-100) that is traceable to
  the scoring function, and every target row has the raw evidence fields.

## Known pitfalls (learned the hard way)

- **Rating is not always in visible text** — some locales put it in an
  aria-label attribute (`5.0 / 5`); parse both.
- **Too-specific queries return 0** — a query phrased like
  "<vertical> <subarea>" can return nothing while "<vertical>" works.
  Keep queries broad, filter locally.
- **Lazy-loaded results** — need explicit scroll of the results feed; pass
  element handles, not locators.
- **Timeout safety** — a full multi-vertical sweep exceeds most shell
  timeouts; run per-vertical and merge incrementally.
- **Analysts bias**: family-diverse models disagree on purpose — when the
  plan is built by a panel, record who said what, don't average to mush.

## What to do when something's off

- Crawl empty → broaden query / check source reachability (the free source
  wants a User-Agent header).
- Report feels wrong → re-check scoring inputs, don't adjust the output.
- Panel disagreement → surface the disagreement as a decision, then decide
  with the evidence table — the report must say *which* evidence won.

---
*Skill pattern: evidence → rank → plan → spar. The value is the discipline,
not the scraping.*