Active Research

Active Research

Analyze any topic, domain, or paper and generate a beautiful HTML report using Actionbook Browser — featuring SPA-aware navigation, network idle detection, batch operations, and intelligent page analysis.

Enhanced Browser Capabilities

Usage

/active-research <topic>
/active-research <topic> --output ./reports/my-report.json

Or simply tell Claude: "Research XXX and generate a report"

Parameters

Topic Detection

Architecture

┌──────────┐     ┌──────────────┐     ┌──────────────┐     ┌──────────┐
│  Claude   │────▶│  Actionbook  │────▶│  Web Pages   │────▶│ Extract  │
│  Code     │     │   Browser    │     │  (multiple)  │     │ Content  │
└──────────┘     └──────────────┘     └──────────────┘     └─────┬────┘
      │           │ wait-idle    │     │ SPA / dynamic │           │
      │           │ batch ops    │     │ protected     │           │
      │           │ --stealth    │     │ mobile-only   │           │
      │           │ snapshot     │     └───────────────┘           │
      │           └──────────────┘                                 │
      │          ┌──────────────┐     ┌──────────────┐            │
      ├─────────▶│  Actionbook  │     │ arXiv Adv.   │            │
      │          │  search/get  │────▶│ Search Form  │───────────▶│
      │          │  (selectors) │     │ (40+ fields) │            │
      │          └──────────────┘     └──────────────┘            │
      │                                                            │
┌──────────┐     ┌──────────────┐     ┌──────────────┐            │
│  Open in │◀────│   json-ui    │◀────│  Write JSON  │◀───────────┘
│  Browser │     │   render     │     │  Report      │  Synthesize
└──────────┘     └──────────────┘     └──────────────┘

MUST USE Actionbook CLI

Always use actionbook browser commands for web browsing. NEVER use any other method to access the web:

• NEVER use curl, wget, httpie, or any HTTP CLI tool via bash
• NEVER use python -c "import requests" or any scripting-language HTTP library via bash
• NEVER use WebFetch or WebSearch tools
• ONLY use actionbook browser and actionbook search/actionbook get commands

If you need web content, the PREFERRED path is: actionbook browser fetch <url> --format text --json (one-shot). For interactive multi-step workflows, use: actionbook browser open <url> → actionbook browser wait-idle → actionbook browser text.

Browser Flags — Research Defaults

CRITICAL: Always use these flags when opening the browser for research.

# PREFERRED: One-shot fetch (I1) — handles open+wait+extract+close automatically
actionbook --block-images --rewrite-urls browser fetch "<url>" --format text --json

# For interactive multi-step workflows, use explicit open:
actionbook --block-images --auto-dismiss-dialogs --no-animations --rewrite-urls browser open "<url>"

For sites with anti-bot protection, add --stealth:

actionbook --block-images --auto-dismiss-dialogs --no-animations --stealth --rewrite-urls browser open "<url>"

Navigation Pattern — ALWAYS Follow

Option A: One-shot fetch (PREFERRED for read-only page extraction):

# Single command: navigate → wait (domain-aware) → extract → close
actionbook --block-images --rewrite-urls browser fetch "<url>" --format text --json
# For static pages (Wikipedia, docs, blogs), add --lite to skip browser entirely:
actionbook --rewrite-urls browser fetch "<url>" --format text --lite --json
# For accessibility tree:
actionbook --block-images --rewrite-urls browser fetch "<url>" --format snapshot --max-tokens 2000 --json

Option B: Interactive multi-step pattern (for forms, clicks, multi-page flows):

# Step 1: Navigate
actionbook browser open "<url>"          # or: goto, click a link

# Step 2: Wait for load (MANDATORY in v2)
actionbook browser wait-idle             # Wait for fetch/XHR to settle

# Step 3: Extract content
actionbook browser text [selector]       # Extract text
# OR
actionbook browser snapshot --filter interactive --max-tokens 500  # Understand page structure

Why wait-idle is critical:

• SPAs (React, Vue, Next.js) load content via fetch/XHR after initial HTML
• Without waiting, text returns empty or incomplete content
• wait-idle monitors all pending network requests, waits until quiet for 500ms

For pages that load content dynamically after network settles:

actionbook browser wait-idle
actionbook browser wait-fn "document.querySelector('.results')"    # Wait for specific element
actionbook browser text ".results"

Complete Workflow

REMINDER: Every web access in this workflow MUST use actionbook browser commands. Using curl, wget, python requests, or any other HTTP tool is strictly forbidden. The bash tool should ONLY be used for actionbook CLI commands and local file operations (json-ui render, open).

Step 1: Plan Search Strategy

Based on the topic, generate 5-8 search queries from different angles:

• Core definition / overview
• Latest developments / news
• Technical details / implementation
• Comparisons / alternatives
• Expert opinions / analysis
• Use cases / applications

Search order — ALWAYS query Actionbook API first, then search:

Step 2: Query Actionbook API for Selectors (ALWAYS DO THIS FIRST)

BEFORE browsing any URL, query Actionbook's indexed selectors.

# Search for indexed actions by domain
actionbook search "<keywords>" -d "<domain>"

# Get detailed selectors for a specific page
actionbook get "<domain>:/<path>:<area>"

Pre-indexed sites useful for research:

For any URL you plan to visit, run actionbook search "<keywords>" -d "<domain>" to check if it's indexed.

Step 3: arXiv Search (URL-First, Form as Backup)

LESSON LEARNED: arXiv form submission via browser automation is unreliable. Use URL-based search as the PRIMARY method.

Option A: URL-based search (PRIMARY — most reliable):

# Simple keyword search
actionbook --block-images --auto-dismiss-dialogs --no-animations browser open "https://arxiv.org/search/?query=large+language+model+agent&searchtype=all"
actionbook browser wait-idle
actionbook browser text "#main-container"

# Advanced URL search with filters
# searchtype: all, title, author, abstract
# start: result offset (0, 50, 100, ...)
actionbook browser open "https://arxiv.org/search/?query=Rust+machine+learning&searchtype=all&start=0"
actionbook browser wait-idle
actionbook browser text "#main-container"

Search strategy: Start broad, then narrow:

1. First search: broad terms (e.g., "Rust" "machine learning") — aim for 50+ results
2. If too few results (< 10): broaden further, remove date/category filters
3. If too many results (> 200): add more specific terms, use searchtype=title
4. Try 2-3 different query angles (e.g., framework names, use cases, benchmarks)

Option B: Form interaction via batch (BACKUP — use if URL search is insufficient):

# Open arXiv with research flags
actionbook --block-images --auto-dismiss-dialogs --no-animations browser open "https://arxiv.org/search/advanced"
actionbook browser wait-idle

# Use batch for form — fewer round-trips, more reliable
cat <<'EOF' | actionbook browser batch --delay 150
{
  "actions": [
    {"kind": "click", "selector": "#terms-0-field"},
    {"kind": "click", "selector": "option[value='title']"},
    {"kind": "type", "selector": "#terms-0-term", "text": "large language model agent"},
    {"kind": "click", "selector": "#classification-computer_science"},
    {"kind": "click", "selector": "#date-filter_by-3"},
    {"kind": "type", "selector": "#date-from_date", "text": "2025-01-01"},
    {"kind": "type", "selector": "#date-to_date", "text": "2026-02-23"},
    {"kind": "click", "selector": "button:has-text('Search'):nth(2)"}
  ],
  "stopOnError": true
}
EOF
actionbook browser wait-idle
actionbook browser text "#main-container"

# If batch form submission fails (page shows form again instead of results):
# → Fall back to Option A URL-based search immediately
# → Do NOT retry the form — it wastes time

arXiv search capabilities (from indexed selectors — for Option B):

Step 4: Supplement with Google / Bing Search

# Search via Google (with wait-idle for SPA results)
actionbook browser open "https://www.google.com/search?q=<encoded_query>"
actionbook browser wait-idle
actionbook browser text "#search"

# Or search via Bing
actionbook browser open "https://www.bing.com/search?q=<encoded_query>"
actionbook browser wait-idle
actionbook browser text "#b_results"

Parse search results to extract URLs. For each discovered URL, query Actionbook API to check if indexed.

CRITICAL: URL Handling Rules (Learned from Production Use)

1. NEVER manually construct URLs from search snippets. Many Google snippet URLs are truncated or reformatted. Instead:

   • Use actionbook browser snapshot --filter interactive to find actual link elements
   • Click the link directly: actionbook browser click "a[href*='domain.com']"
   • Or extract href from snapshot refs

2. Expect 20-30% of URLs to be dead. In practice, ~5 out of 20 URLs return 404. Handle this:

   actionbook browser open "<url>"
   actionbook browser wait-idle
   # Check if the page is a 404 or error page
   actionbook browser wait-fn "!document.title.includes('404') && !document.title.includes('Not Found')" --timeout 3000
   # If timeout → page is dead, skip immediately. Do NOT retry.
   

3. Salvage info from Google snippets. If a URL is dead but the Google snippet had useful info:

   • The snippet text you already extracted IS valid data
   • Use it in the report with a note that the source is no longer available
   • Search for the same content on alternative sites (archive.org, cached versions)

4. Use 4+ diverse search queries. Don't rely on one search angle:

   • Query 1: Core topic overview (e.g., "Rust AI ecosystem 2026")
   • Query 2: Specific frameworks/tools (e.g., "Candle vs Burn Rust ML framework")
   • Query 3: Use cases/benchmarks (e.g., "Rust LLM inference performance benchmark")
   • Query 4: Recent news/developments (e.g., "Rust machine learning latest 2026")
   • Query 5: Community/ecosystem (e.g., "Rust AI agent framework comparison")

Step 5: Deep Read Sources

PREFERRED: Use browser fetch for one-shot page extraction (handles wait + extract + cleanup):

# Quick text extraction (most common)
actionbook --block-images --rewrite-urls browser fetch "<url>" --format text --json

# Static pages (Wikipedia, docs, blogs) — skip browser entirely
actionbook --rewrite-urls browser fetch "<url>" --format text --lite --json

# Page structure analysis
actionbook --block-images --rewrite-urls browser fetch "<url>" --format snapshot --max-tokens 2000 --json

# With token budget for LLM context management
actionbook --block-images --rewrite-urls browser fetch "<url>" --format text --max-tokens 4000 --json

For interactive workflows (forms, clicks), fall back to multi-step:

actionbook browser open "<url>"
actionbook browser wait-idle                    # MANDATORY: wait for network
actionbook browser text                         # Full page text (fallback)
actionbook browser text "<selector>"            # Use Actionbook selector if indexed

If page content seems incomplete, debug:

# Check for JS errors that might block rendering
actionbook browser console --level error

# Check if a specific element exists
actionbook browser wait-fn "document.querySelector('.content')" --timeout 5000

# Inspect element properties
actionbook browser info ".content"

For arXiv papers, try sources in this order:

# 1. arXiv abstract (most reliable) — use fetch
actionbook --block-images browser fetch "https://arxiv.org/abs/<arxiv_id>" --format text --json

# 2. HuggingFace papers page
actionbook --block-images browser fetch "https://huggingface.co/papers/<arxiv_id>" --format text --json

# 3. ar5iv HTML (structured, but fails on new papers) — use --lite for static HTML
actionbook browser fetch "https://ar5iv.org/html/<arxiv_id>" --format text --lite --json
# NOTE: if content too short, ar5iv didn't render. Fall back.

# 4. GitHub repo (from search results) — use fetch
actionbook --block-images browser fetch "<github_repo_url>" --format text --json

For protected sites (Cloudflare, bot detection) — use interactive mode with stealth:

actionbook --stealth --block-images --auto-dismiss-dialogs --rewrite-urls browser open "<protected_url>"
actionbook browser wait-idle
actionbook browser text

For mobile-only content:

actionbook browser emulate iphone-14
actionbook browser open "<url>"
actionbook browser wait-idle
actionbook browser text

For Google Scholar (indexed by Actionbook):

actionbook browser open "https://scholar.google.com"
actionbook browser wait-idle
actionbook browser click "#gs_hdr_tsi"
actionbook browser type "#gs_hdr_tsi" "<query>"
actionbook browser click "#gs_hdr_tsb"
actionbook browser wait-idle
actionbook browser text "#gs_res"

For unindexed sites, use snapshot to discover structure:

actionbook --block-images browser fetch "<url>" --format snapshot --max-tokens 800 --json

Step 6: Synthesize Findings

Organize collected information into a coherent report:

1. Overview / Executive Summary
2. Key Findings
3. Detailed Analysis
4. Supporting Data / Evidence
5. Implications / Significance
6. Sources

Step 7: Generate json-ui JSON Report

Write a JSON file following the @actionbookdev/json-ui schema. Use the Write tool.

Output path: ./output/<topic-slug>.json (or user-specified --output path)

Step 8: Render HTML

CRITICAL: You MUST try ALL fallback methods before giving up. Do NOT stop at the first failure.

IMPORTANT: Always use ABSOLUTE paths for JSON_FILE and HTML_FILE.

Try each method one by one until one succeeds:

# Method 1: Monorepo absolute path (most reliable if inside actionbook project)
node "$(git rev-parse --show-toplevel)/packages/json-ui/dist/cli.js" render /absolute/path/to/report.json -o /absolute/path/to/report.html

# Method 2: Global install (if user ran: cd packages/json-ui && npm link)
json-ui render /absolute/path/to/report.json -o /absolute/path/to/report.html

# Method 3: npx (if published to npm)
npx @actionbookdev/json-ui render /absolute/path/to/report.json -o /absolute/path/to/report.html

NEVER give up silently. If all methods fail, tell the user:

1. The JSON report is saved at <path>
2. To enable HTML rendering, run: cd <actionbook-repo>/packages/json-ui && npm link

Step 9: Open in Browser

# macOS
open <report.html>

# Linux
xdg-open <report.html>

Step 10: Close Browser

Always close the browser when done:

actionbook browser close

Error Recovery Patterns

Intelligent error recovery using advanced browser capabilities:

Pattern: Page Load Failure

# 1. Open page
actionbook browser open "<url>"
actionbook browser wait-idle --timeout 15000

# 2. Check for JS errors
actionbook browser console --level error
# If errors found → page is broken, skip to next source

# 3. Check if content rendered
actionbook browser wait-fn "document.body.innerText.length > 100" --timeout 5000
# If timeout → content didn't render, try fallback

Pattern: Selector Not Found

# 1. Use snapshot to discover actual page structure
actionbook browser snapshot --filter interactive --max-tokens 800

# 2. Or inspect a specific area
actionbook browser info "<parent_selector>"
# Returns: suggested selectors, visibility, tag info

# 3. Adjust selector and retry

Pattern: Anti-Bot Detection

# 1. If initial load returns CAPTCHA or access denied:
actionbook browser close

# 2. Reopen with stealth
actionbook --stealth --no-animations --auto-dismiss-dialogs browser open "<url>"
actionbook browser wait-idle

# 3. If still blocked, rotate fingerprint
actionbook browser fingerprint rotate --os windows
actionbook browser open "<url>"
actionbook browser wait-idle

Pattern: SPA Content Not Loading

# 1. Wait for network
actionbook browser wait-idle --idle-time 1000 --timeout 15000

# 2. Wait for specific element
actionbook browser wait-fn "document.querySelector('.results')" --timeout 10000

# 3. If still empty, check console
actionbook browser console --level error

# 4. Try clicking a loading trigger
actionbook browser snapshot --filter interactive --max-tokens 300
# Look for "Load More", "Show Results", etc.

Full Error Handling Reference

IMPORTANT: Always run actionbook browser close before finishing, even on errors.

Feature Usage Checklist

Before finalizing research, verify you used these capabilities:

Common mistakes to avoid:

• Using manual open → wait-idle → text → close when browser fetch does it in one command
• Not using --lite for static pages (Wikipedia, docs, blogs) — wastes 5-10s on browser startup
• Not using --rewrite-urls — x.com and reddit.com have aggressive anti-bot that blocks scraping
• Forgetting wait-idle after navigation in interactive mode (content appears empty)
• Not using batch for form interactions (slow, unreliable)
• Retrying a dead URL instead of skipping it
• Constructing URLs manually from search snippets instead of clicking links
• Using only one search query angle (always use 4+ diverse queries)
• Not checking for 404 pages before extracting content

json-ui Report Template

IMPORTANT: Always include BrandHeader and BrandFooter.

{
  "type": "Report",
  "props": { "theme": "auto" },
  "children": [
    {
      "type": "BrandHeader",
      "props": {
        "badge": "Deep Research Report",
        "poweredBy": "Actionbook"
      }
    },
    {
      "type": "Section",
      "props": { "title": "Overview", "icon": "paper" },
      "children": [
        {
          "type": "Prose",
          "props": {
            "content": "Overview of the topic..."
          }
        }
      ]
    },
    {
      "type": "Section",
      "props": { "title": "Key Findings", "icon": "star" },
      "children": [
        {
          "type": "ContributionList",
          "props": {
            "items": [
              {
                "badge": "Finding",
                "title": "...",
                "description": "..."
              }
            ]
          }
        }
      ]
    },
    {
      "type": "Section",
      "props": { "title": "Detailed Analysis", "icon": "bulb" },
      "children": [
        {
          "type": "Prose",
          "props": { "content": "..." }
        }
      ]
    },
    {
      "type": "Section",
      "props": { "title": "Key Metrics", "icon": "chart" },
      "children": [
        {
          "type": "MetricsGrid",
          "props": { "metrics": [], "cols": 3 }
        }
      ]
    },
    {
      "type": "Section",
      "props": { "title": "Sources", "icon": "link" },
      "children": [
        {
          "type": "LinkGroup",
          "props": { "links": [] }
        }
      ]
    },
    {
      "type": "BrandFooter",
      "props": {
        "timestamp": "YYYY-MM-DDTHH:MM:SSZ",
        "attribution": "Powered by Actionbook",
        "disclaimer": "This report was generated by AI using web sources. Verify critical information independently."
      }
    }
  ]
}

Paper Report Template (for arXiv papers)

When analyzing academic papers, use a richer template with:

• PaperHeader (title, arxivId, date, categories)
• AuthorList (authors with affiliations)
• Abstract (with keyword highlights)
• ContributionList (key contributions)
• MethodOverview (step-by-step method)
• ResultsTable (experimental results)
• Formula (key equations, LaTeX)
• Figure (paper figures from ar5iv)

Available json-ui Components

json-ui Known Pitfalls

Text Fields

All text fields should use plain English strings.

{ "title": "Key Findings" }

Note: MetricsGrid props value and suffix, and Table row cell values must always be plain strings.

Academic Paper Support

arXiv Papers

ar5iv.org HTML (preferred for reading, but often incomplete for papers < 3 months old):

Recommended approach: Use wait-idle + wait-fn to verify ar5iv content loaded:

actionbook browser open "https://ar5iv.org/html/<arxiv_id>"
actionbook browser wait-idle --timeout 15000
actionbook browser wait-fn "document.body.innerText.length > 5000" --timeout 10000
# If wait-fn times out → content didn't render, fall back to other sources

Recommended Source Priority

Other Academic Sources

• Google Scholar (scholar.google.com) — Actionbook indexed
• Semantic Scholar (semanticscholar.org)
• Papers With Code (paperswithcode.com)
• Conference proceedings sites

Quality Guidelines

1. Breadth: Research from at least 3-5 diverse sources
2. Depth: Read full articles, not just snippets
3. Accuracy: Cross-reference facts across sources
4. Structure: Use appropriate json-ui components for each content type
5. Attribution: Always include source links in the report
6. Freshness: Prefer recent sources when relevance is equal