# Crawl To Markdown

> Use when web.run fails to parse a webpage and you need raw Markdown via Crawl4AI.

- Skill: `majiayu000/crawl-to-markdown` (Agent Skill, multi-file: 2 files)
- Install (CLI): `npx skillmds add majiayu000/crawl-to-markdown`
- Raw SKILL.md: https://api.skillmd.com/api/skills/majiayu000/crawl-to-markdown/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Docs & Writing
- Author: majiayu000 (https://skillmd.com/u/majiayu000)
- Updated: 2026-09-09
- Page: https://skillmd.com/skills/majiayu000/crawl-to-markdown

---


Use this skill only after the built-in `web.run` tool fails to return usable content for a page.

## What this skill does

- Crawls one or more URLs with Crawl4AI.
- Returns raw Markdown for each URL.

## How to run

This script uses uv inline metadata. If `uv` is available just run:

```bash
uv run scripts/crawl_to_markdown.py https://example.com
```

Multiple URLs:

```bash
uv run scripts/crawl_to_markdown.py https://example.com https://example.org
```

Or pipe a newline-delimited list:

```bash
printf "https://example.com\nhttps://example.org\n" | uv run scripts/crawl_to_markdown.py
```

## Troubleshooting

### Crawl succeeds but output is blank

Some sites don't match the default `--selector` and can return empty/whitespace Markdown.
The script will retry without CSS selection automatically, but you can also force it explicitly:

```bash
uv run scripts/crawl_to_markdown.py --selector '' https://example.com
```

### uv cache permission errors

If `uv` fails accessing its cache directory (e.g. `~/.cache/uv`), either point it at a
writable cache directory:

```bash
uv --cache-dir .uv-cache run scripts/crawl_to_markdown.py https://example.com
```

Or avoid the cache entirely:

```bash
uv --no-cache run scripts/crawl_to_markdown.py https://example.com
```

## Output format

For each URL, the script prints a header and raw Markdown content:

```
URL: https://example.com
<raw markdown>
---
```

If a crawl fails, the output includes an error block for that URL.