Competitor Profiling

Saving Raw Data

Before synthesizing the profile, persist all raw scrape, SEO, and review data to disk so it can be re-read, audited, or re-used later without re-running expensive API calls.

Directory layout (relative to project root):

competitor-profiles/
├── raw/
│   └── <competitor-slug>/
│       └── <YYYY-MM-DD>/
│           ├── scrapes/    # one .md file per scraped page (homepage.md, pricing.md, ...)
│           ├── seo/        # one .json file per DataForSEO call (backlinks-summary.json, ranked-keywords.json, ...)
│           └── reviews/    # one .md or .json file per review source (g2.md, capterra.md, ...)
├── <competitor-slug>.md    # final synthesized profile
└── _summary.md             # cross-competitor summary

Rules:

• <competitor-slug> is lowercase, hyphenated (e.g. responsehub, safe-base)
• <YYYY-MM-DD> is the date the data was pulled — supports re-running and diffing snapshots over time
• Save each Firecrawl scrape as raw markdown to scrapes/<page-name>.md
• Save each DataForSEO response as raw JSON to seo/<endpoint-name>.json
• Save each review source to reviews/<source>.md (cleaned text) or .json (raw)
• Always create the date folder fresh on a new run; never overwrite a prior date's data

The synthesized profile (<competitor-slug>.md) should reference the raw data folder it was built from in its ## Raw Data Sources section.

Research Process

Phase 1: Site Scraping (Firecrawl)

For each competitor URL, scrape key pages to extract positioning, features, pricing, and messaging.

Step 1: Map the site

Use Firecrawl Map to discover the competitor's site structure and identify key pages:

firecrawl_map → competitor URL

From the map, identify and prioritize these page types:

• Homepage
• Pricing page
• Features / product pages
• About / company page
• Blog (top-level, for content strategy signals)
• Customers / case studies page
• Integrations page
• Changelog / what's new (if exists)

Step 2: Scrape key pages

Use Firecrawl Scrape on each identified page:

firecrawl_scrape → each key page URL

Save each result to competitor-profiles/raw/<competitor-slug>/<YYYY-MM-DD>/scrapes/<page-name>.md before extracting fields.

Extract from each page:

Step 3: Scrape competitor reviews (optional but high-value)

Use Firecrawl Scrape or Firecrawl Search to find:

• G2 reviews page for the competitor
• Capterra reviews page
• Product Hunt launch page
• TrustRadius profile

Save each scraped review page to competitor-profiles/raw/<competitor-slug>/<YYYY-MM-DD>/reviews/<source>.md. Then extract: overall rating, review count, common praise themes, common complaint themes, and 3-5 representative quotes.

Phase 2: SEO & Market Data (DataForSEO)

Use DataForSEO MCP tools to gather quantitative competitive intelligence. Save each raw response as JSON to competitor-profiles/raw/<competitor-slug>/<YYYY-MM-DD>/seo/<endpoint-name>.json before parsing it into the profile. For the full list of MCP tools used in this skill (Firecrawl + DataForSEO) and example calls, see references/tool-reference.md.

Domain Authority & Backlinks

Use backlinks_summary to get:

• Domain rank / authority score
• Total backlinks
• Referring domains count
• Spam score

Use backlinks_referring_domains for:

• Top referring domains (quality signals)
• Link acquisition patterns

Keyword & Traffic Intelligence

Use dataforseo_labs_google_ranked_keywords to get:

• Total organic keywords ranking
• Keywords in top 3, top 10, top 100
• Estimated organic traffic

Use dataforseo_labs_google_domain_rank_overview for:

• Domain-level organic metrics
• Estimated traffic value
• Top keywords by traffic

Use dataforseo_labs_google_keywords_for_site to discover:

• What keywords they target
• Content gaps vs. your site

Competitive Positioning Data

Use dataforseo_labs_google_competitors_domain to find:

• Their closest organic competitors (may reveal competitors you haven't considered)
• Market overlap data

Use dataforseo_labs_google_relevant_pages to find:

• Their highest-traffic pages
• Content that drives the most organic value

Phase 3: Synthesis

Combine scraped content with SEO data to build the profile. Cross-reference claims (e.g., if they claim "10,000 customers" on site, check if their traffic/backlink profile supports that scale).

Output Format

Profile Document Structure

Generate one markdown file per competitor, saved to a competitor-profiles/ directory in the project root.

Filename: competitor-profiles/[competitor-name].md

For the full profile and summary templates: See references/templates.md

Each profile follows this structure:

# [Competitor Name] — Competitor Profile

**URL**: [website]
**Generated**: [date]
**Depth**: [quick scan / deep profile]

---

## At a Glance

| Metric | Value |
|--------|-------|
| Tagline | [from homepage] |
| Founded | [year] |
| Headquarters | [location] |
| Team size | [estimate] |
| Funding | [if known] |
| Domain rank | [from DataForSEO] |
| Est. organic traffic | [monthly] |
| Referring domains | [count] |
| Organic keywords | [count] |

---

## Positioning & Messaging

**Primary value proposition**: [headline + subheadline from homepage]

**Target audience**: [who they're speaking to, based on copy analysis]

**Positioning angle**: [how they position — e.g., "simplicity-first," "enterprise-grade," "all-in-one"]

**Key messaging themes**:
- [theme 1 — with source page]
- [theme 2]
- [theme 3]

---

## Product & Features

### Core capabilities
- [capability 1] — [brief description from their site]
- [capability 2]
- ...

### Notable differentiators
- [what they emphasize as unique]

### Integrations
- [count] integrations
- Key: [list top 5-10]

### Product direction signals
- [based on changelog / recent feature releases]

---

## Pricing

| Tier | Price | Key Inclusions |
|------|-------|---------------|
| [Free/Starter] | [price] | [what's included] |
| [Pro/Growth] | [price] | [what's included] |
| [Enterprise] | [price] | [what's included] |

**Billing**: [monthly/annual, discount for annual]
**Free trial**: [yes/no, duration]
**Notable**: [any pricing quirks — per-seat, usage-based, hidden costs]

---

## Customers & Social Proof

**Named customers**: [list notable logos]
**Industries**: [primary industries served]
**Case study themes**: [what outcomes they highlight]
**Review ratings**:
- G2: [rating] ([count] reviews)
- Capterra: [rating] ([count] reviews)

---

## SEO & Content Strategy

**Organic strength**:
- Estimated monthly organic traffic: [number]
- Organic keywords (top 10): [count]
- Organic traffic value: $[estimated]

**Top organic pages** (by estimated traffic):
1. [page URL] — [keyword] — [est. traffic]
2. [page URL] — [keyword] — [est. traffic]
3. [page URL] — [keyword] — [est. traffic]

**Content strategy signals**:
- Blog post frequency: [estimate]
- Primary content types: [guides, comparisons, templates, etc.]
- Content focus areas: [topics they invest in]

**Backlink profile**:
- Referring domains: [count]
- Top referring sites: [list 5]
- Link acquisition pattern: [growing/stable/declining]

---

## Strengths & Weaknesses

### Strengths
- [strength 1 — with evidence source]
- [strength 2]
- [strength 3]

### Weaknesses
- [weakness 1 — with evidence source]
- [weakness 2]
- [weakness 3]

---

## Competitive Implications for [Your Product]

**Where they're strong vs. us**: [areas where this competitor has an advantage]

**Where we're strong vs. them**: [areas where you have an advantage]

**Opportunities**: [gaps in their offering or positioning we can exploit]

**Threats**: [areas where they're improving or gaining ground]

---

## Raw Data Sources

- Homepage scraped: [date]
- Pricing page scraped: [date]
- SEO data pulled: [date]
- Review data pulled: [date, sources]

Summary Document

After profiling all competitors, generate a competitor-profiles/_summary.md that includes:

1. Competitor landscape overview — one paragraph summarizing the competitive field
2. Comparison table — key metrics side by side for all profiled competitors
3. Positioning map — where each competitor sits (e.g., simple↔complex, cheap↔premium)
4. Key takeaways — 3-5 strategic observations from the research
5. Gaps and opportunities — where the market is underserved

Quick Scan vs. Deep Profile

Quick Scan (faster, lower cost)

• Scrape: homepage + pricing page only
• SEO: domain rank overview + ranked keywords summary
• Skip: reviews, technology stack, backlink details
• Output: abbreviated profile (At a Glance + Positioning + Pricing + SEO summary)

Deep Profile (comprehensive)

• Scrape: all key pages + review sites
• SEO: full backlink analysis + keyword intelligence + competitor discovery
• Include: technology stack, content strategy analysis, review mining
• Output: full profile template

Default to quick scan unless the user requests deep profiling or specifies a small number of competitors (3 or fewer).

Handling Multiple Competitors

When profiling more than one competitor:

1. Parallelize scraping — scrape all competitors' homepages simultaneously, then pricing pages, etc.
2. Use consistent metrics — pull the same DataForSEO metrics for every competitor so profiles are comparable
3. Build the summary last — after all individual profiles are complete
4. Prioritize by relevance — if the user has 10+ competitors, suggest profiling the top 5 first based on domain overlap or market similarity

Updating Profiles

Profiles are snapshots. When updating:

• Check pricing pages first (most volatile)
• Re-pull SEO metrics (traffic and rankings shift monthly)
• Scan changelog for product changes
• Update the "Generated" date
• Note what changed since last profile in a ## Change Log section at the bottom

Task-Specific Questions

Only ask if not answered by context or input:

1. What competitor URLs should I profile?
2. Quick scan or deep profile?
3. Any specific dimensions to focus on (pricing, SEO, positioning)?
4. Should I compare findings against your product?

Related Skills

• competitor-alternatives: For creating comparison/alternative pages from these profiles
• customer-research: For mining reviews and community sentiment in depth
• content-strategy: For using competitor content gaps to plan your own content
• seo-audit: For auditing your own site relative to competitors
• sales-enablement: For turning profiles into battle cards and sales collateral
• paid-ads: For analyzing competitor ad strategies
• pricing-strategy: For deeper pricing analysis informed by competitor profiles