Content Research

Content Research Skill

Installation

npx clawhub@latest install content-research-mcbai

Search the web for trending articles, news, and content sources on any topic. This skill powers the MCB AI content research pipeline — finding, filtering, scoring, and organizing source material for content creation.

Search Strategy: Brave + Tavily Dual-Engine

This skill uses TWO search providers in parallel for maximum coverage:

• Brave Search — via web_search tool (built-in OpenClaw tool)
• Tavily — via direct API call using TAVILY_API_KEY from ~/.openclaw/.env

Tavily API Call

POST https://api.tavily.com/search
Headers: Content-Type: application/json
Body:
{
  "api_key": "<TAVILY_API_KEY>",
  "query": "<query>",
  "search_depth": "advanced",
  "include_answer": false,
  "include_raw_content": false,
  "max_results": 10,
  "topic": "news"   // use "general" for non-news searches
}

Run Tavily via exec with PowerShell:

$body = @{
  api_key = $env:TAVILY_API_KEY
  query = "<query>"
  search_depth = "advanced"
  include_answer = $false
  include_raw_content = $false
  max_results = 10
  topic = "news"
} | ConvertTo-Json

Invoke-RestMethod -Uri "https://api.tavily.com/search" -Method Post -ContentType "application/json" -Body $body

Fallback Logic

• Run Brave (web_search) and Tavily in parallel
• If Brave fails → use Tavily results only
• If Tavily fails → use Brave results only
• If both succeed → merge and deduplicate by URL

When to Use

• User wants to research a topic before writing content
• User needs to find recent articles, news, or data about a subject
• User wants to discover trending content sources for LinkedIn/social media
• User needs to curate sources for a toplist, POV, case study, or how-to post

Core Workflow

Step 1: Understand the Research Request

Extract from the user's message:

1. Topic — the subject to research (required)
2. Source filter — where to search (default: all sources)
   • all — All web sources
   • news — News publications only
   • linkedin — LinkedIn posts/articles (append site:linkedin.com)
   • youtube — YouTube videos (append site:youtube.com)
   • blogs — Blog posts and articles (append blog OR article OR guide)
3. Freshness — how recent (default: past month for web, past week for news)
4. Count — how many results to return (default: 10-15)

If the user doesn't specify these, use sensible defaults and mention what you chose.

Step 2: Execute Dual Search (Brave + Tavily)

Run BOTH providers. Each provider runs TWO queries when possible.

Brave Search (web_search tool)

Query 1 — Web:

Query: {topic} {source_filter_query}
count: 10
freshness: month

Query 2 — News:

Query: {topic} news
count: 10
freshness: week

Tavily Search (exec PowerShell)

Query 1 — General:

$env:TAVILY_API_KEY = (Get-Content "$env:USERPROFILE\.openclaw\.env" | Select-String "TAVILY_API_KEY" | ForEach-Object { $_ -replace "TAVILY_API_KEY=", "" })

$body = @{
  api_key = $env:TAVILY_API_KEY.Trim()
  query = "{topic}"
  search_depth = "advanced"
  include_answer = $false
  include_raw_content = $false
  max_results = 10
  topic = "general"
} | ConvertTo-Json

Invoke-RestMethod -Uri "https://api.tavily.com/search" -Method Post -ContentType "application/json" -Body $body

Query 2 — News:

# Same as above but topic = "news"

Step 3: Merge and Deduplicate

1. Collect all results from Brave (web + news) and Tavily (general + news)
2. Deduplicate by URL — keep one copy per URL, prefer Tavily version (richer content)
3. Sort by relevance:
   • News articles first (most time-sensitive)
   • Then by freshness (most recent first)
4. Limit to requested count (default 15)
5. Label source engine in metadata: [Brave] or [Tavily]

Step 4: Process and Organize Results

For each result, extract and structure:

Article:
  - Title: [article title]
  - Source: [publication/website name]
  - URL: [full URL]
  - Date: [relative date, e.g. "2 hours ago", "3 days ago"]
  - Summary: [description/snippet from search]
  - Type: [News / Blog / Report / Video / LinkedIn]
  - Tag: [auto-detected tag, see Tag Rules below]
  - Engine: [Brave / Tavily / Both]

Source Name Extraction

Clean the hostname to a readable name:

• Remove www. prefix
• Remove .com, .org, .net, .io, .co suffixes
• Map known domains: techcrunch → TechCrunch, crunchbase → Crunchbase, forbes → Forbes, bloomberg → Bloomberg, reuters → Reuters, etc.

Auto-Tag Rules

Scan title + summary and apply the FIRST matching tag:

Step 5: Present Results

Present the organized results in a clear, scannable format:

## Research Results: "{topic}"
Found {N} articles from {sources_count} sources
Sources: Brave ({brave_count}) + Tavily ({tavily_count}) → merged {total} unique

### 📰 News
1. **{title}** — {source} ({date}) [{engine}]
   {summary}
   🏷️ {tag} | 🔗 {url}

### 📝 Articles & Blogs
2. **{title}** — {source} ({date}) [{engine}]
   {summary}
   🏷️ {tag} | 🔗 {url}

...

Then ask the user which articles they want to use for content creation. If the user wants to proceed to writing, hand off to the content-writer skill with the selected articles.

Output Format

Always provide results as a numbered list with:

• Clear title
• Source name and date
• Engine label [Brave] or [Tavily]
• Brief summary (1-2 lines)
• Auto-detected tag
• Source URL

Tips for Better Research

• For funding/startup topics: search for specific company names + "funding" or "series"
• For trend pieces: include year/quarter in the search (e.g., "AI trends Q1 2026")
• For competitive analysis: search for specific company + "vs" or "alternative"
• For LinkedIn content: recent news performs best (past 1-2 weeks)
• Combine multiple source types for richer content
• Tavily search_depth: "advanced" digs deeper — use for complex topics
• If one engine returns fewer results than expected, note it in the summary

Integration with Content Writer

After research, the user typically selects articles and moves to writing. Pass the selected articles to the content-writer skill in this format:

{
  "articles": [
    {
      "title": "Article title",
      "source": "Publication name",
      "url": "https://...",
      "date": "2 days ago",
      "summary": "Brief description",
      "tag": "AI",
      "engine": "Tavily"
    }
  ]
}

See references/source-filters.md for detailed source filter configurations.