crw-search — web search via crw's own search backend
When to use
- You have a question or topic, not a URL. Get candidate URLs first, then
scrape the one that looks right.
- Step 1 in the crw ladder: most searches should end at
step 2 (crw-scrape) — pick the best result URL
and scrape it for full content.
- Token-heavy context? Pipe through a subprocess filter instead of dumping
raw JSON. See crw-dynamic-search.
- The search backend is self-hosted and free — no API key, no per-query billing,
no usage cap. Queries never leave your infrastructure in embedded/local mode.
Quick start
CLI (binary on PATH):
crw search "rust async http client" # text output
crw search "site:docs.rs tokio" --json --fields title,url,snippet --limit 5
crw search "CVE-2024-1234" --category news --time-range week
crw search "climate policy 2025" --json -o .crw/results.json
crw search "rust crates" --language en --limit 20
MCP (inside an agent harness):
crw_search(query="rust async http client", limit=5, lang="en")
crw_search(query="latest CVE nginx", tbs="qdr:w", categories="news")
crw_search(query="openai pricing", scrapeOptions={"formats": ["markdown"]})
REST (drop-in for Firecrawl SDKs — just swap the base URL):
curl -X POST "$CRW_API_URL/v1/search" -H "Authorization: Bearer $CRW_API_KEY" \
-H 'Content-Type: application/json' \
-d '{"query":"rust async http","limit":5,"lang":"en"}'
Options
| Need |
CLI flag |
MCP / REST field |
| Result count |
-l/--limit N (default 10) |
limit (default 5) |
| JSON output |
--json or --format json |
— (always JSON) |
| Field projection |
--fields title,url,snippet |
— |
| Output to file |
-o FILE |
— |
| Filter by category |
--category news|images|videos|general|… |
categories |
| Language |
--language en |
lang |
| Time filter |
--time-range day|week|month|year |
tbs: qdr:h|qdr:d|qdr:w|qdr:m|qdr:y |
| Safe search |
--safesearch 0|1|2 |
— |
| Custom search backend |
--searxng-url URL / $CRW_SEARXNG_URL |
— |
| Group by source |
— |
sources: ["web","news","images"] |
| Scrape results inline |
— (use crw scrape separately) |
scrapeOptions: {formats:["markdown"]} |
--fields available values: title, url, description, snippet,
position, score, category. snippet is an alias for description.
A note on result scores
The search backend is a meta-search aggregator — it merges results from
multiple engines (Google, Bing, DuckDuckGo, etc.) and the score field
reflects internal engine weighting, not a universal relevance measure. Do not
rely on score for ranking or filtering. Use position (1-based rank) or
result order instead — position 1 is the most relevant result surfaced.
Tips
- No results / 403? The search backend needs JSON output enabled in its
config. Run
crw setup --local to spin up a pre-configured sidecar
automatically. Public instances usually block JSON with 403/429.
--fields saves context. --json --fields title,url,snippet --limit 5
is one call; piping to jq is two. Prefer the flag.- Inline scraping via MCP. Pass
scrapeOptions: {formats: ["markdown"]}
to get page content alongside search results in one round-trip. There is no
--scrape CLI flag — use the MCP/REST path for this.
- Time-sensitive queries. Use
--time-range week (CLI) or tbs: "qdr:w"
(MCP/REST) for news, CVEs, releases, or any freshness-sensitive topic.
- After search, scrape the winner.
crw search "…" returns candidates;
crw scrape "<url>" gets the full content. Don't try to read content from
search snippets alone.
- Write large result sets to
.crw/. Never stream a 20-result JSON blob
to stdout into context. Use -o .crw/results.json then jq/grep.
See also
- crw-scrape — scrape the URL you found
- crw-dynamic-search — filter output in a
subprocess to save context (use this on token-heavy tasks)
- crw — hub skill with the full workflow ladder
1---2name: crw-search3description: Search the web with fastCRW and get titles, URLs, and descriptions. Use when you have a question or topic but not a URL — "search for", "find pages about", "look up", "what is", "who is", "latest news on", "find docs for". Own search backend: self-hosted, no API key, no per-query cost, high recall via meta-search aggregation. Step 1 of the crw workflow ladder.4license: AGPL-3.05---67# crw-search — web search via crw's own search backend89## When to use1011- You have a **question or topic**, not a URL. Get candidate URLs first, then12 scrape the one that looks right.13- Step 1 in the [crw ladder](../crw/SKILL.md): most searches should end at14 step 2 ([crw-scrape](../crw-scrape/SKILL.md)) — pick the best result URL15 and scrape it for full content.16- **Token-heavy context?** Pipe through a subprocess filter instead of dumping17 raw JSON. See [crw-dynamic-search](../crw-dynamic-search/SKILL.md).18- The search backend is self-hosted and free — no API key, no per-query billing,19 no usage cap. Queries never leave your infrastructure in embedded/local mode.2021## Quick start2223**CLI** (binary on PATH):24```bash25crw search "rust async http client" # text output26crw search "site:docs.rs tokio" --json --fields title,url,snippet --limit 527crw search "CVE-2024-1234" --category news --time-range week28crw search "climate policy 2025" --json -o .crw/results.json29crw search "rust crates" --language en --limit 2030```3132**MCP** (inside an agent harness):33```34crw_search(query="rust async http client", limit=5, lang="en")35crw_search(query="latest CVE nginx", tbs="qdr:w", categories="news")36crw_search(query="openai pricing", scrapeOptions={"formats": ["markdown"]})37```3839**REST** (drop-in for Firecrawl SDKs — just swap the base URL):40```bash41curl -X POST "$CRW_API_URL/v1/search" -H "Authorization: Bearer $CRW_API_KEY" \42 -H 'Content-Type: application/json' \43 -d '{"query":"rust async http","limit":5,"lang":"en"}'44```4546## Options4748| Need | CLI flag | MCP / REST field |49|------|----------|------------------|50| Result count | `-l/--limit N` (default 10) | `limit` (default 5) |51| JSON output | `--json` or `--format json` | — (always JSON) |52| Field projection | `--fields title,url,snippet` | — |53| Output to file | `-o FILE` | — |54| Filter by category | `--category news\|images\|videos\|general\|…` | `categories` |55| Language | `--language en` | `lang` |56| Time filter | `--time-range day\|week\|month\|year` | `tbs: qdr:h\|qdr:d\|qdr:w\|qdr:m\|qdr:y` |57| Safe search | `--safesearch 0\|1\|2` | — |58| Custom search backend | `--searxng-url URL` / `$CRW_SEARXNG_URL` | — |59| Group by source | — | `sources: ["web","news","images"]` |60| Scrape results inline | — (use crw scrape separately) | `scrapeOptions: {formats:["markdown"]}` |6162**`--fields` available values:** `title`, `url`, `description`, `snippet`,63`position`, `score`, `category`. `snippet` is an alias for `description`.6465## A note on result scores6667The search backend is a **meta-search aggregator** — it merges results from68multiple engines (Google, Bing, DuckDuckGo, etc.) and the `score` field69reflects internal engine weighting, not a universal relevance measure. Do not70rely on `score` for ranking or filtering. Use **`position`** (1-based rank) or71**result order** instead — position 1 is the most relevant result surfaced.7273## Tips7475- **No results / 403?** The search backend needs JSON output enabled in its76 config. Run `crw setup --local` to spin up a pre-configured sidecar77 automatically. Public instances usually block JSON with 403/429.78- **`--fields` saves context.** `--json --fields title,url,snippet --limit 5`79 is one call; piping to `jq` is two. Prefer the flag.80- **Inline scraping via MCP.** Pass `scrapeOptions: {formats: ["markdown"]}`81 to get page content alongside search results in one round-trip. There is no82 `--scrape` CLI flag — use the MCP/REST path for this.83- **Time-sensitive queries.** Use `--time-range week` (CLI) or `tbs: "qdr:w"`84 (MCP/REST) for news, CVEs, releases, or any freshness-sensitive topic.85- **After search, scrape the winner.** `crw search "…"` returns candidates;86 `crw scrape "<url>"` gets the full content. Don't try to read content from87 search snippets alone.88- **Write large result sets to `.crw/`.** Never stream a 20-result JSON blob89 to stdout into context. Use `-o .crw/results.json` then `jq`/`grep`.9091## See also9293- [crw-scrape](../crw-scrape/SKILL.md) — scrape the URL you found94- [crw-dynamic-search](../crw-dynamic-search/SKILL.md) — filter output in a95 subprocess to save context (use this on token-heavy tasks)96- [crw](../crw/SKILL.md) — hub skill with the full workflow ladder
us