html2md
Aggressive HTML-to-markdown converter for AI agents. Mozilla Readability isolates main content, Turndown converts to markdown, then heavy post-processing strips remaining noise.
Full flag reference and advanced examples: references/usage.md
Setup
cd <skill-dir>/scripts
npm install
npm link # makes `html2md` globally available
Requires Node.js 22+.
Quick Start
html2md https://example.com # fetch + convert
html2md --file page.html # local HTML file
cat page.html | html2md --stdin # pipe from stdin
html2md --max-tokens 2000 https://example.com # budget-aware truncation
html2md --no-links https://example.com # strip hrefs, keep text
html2md --json https://example.com # JSON: {title, url, markdown, tokens}
Key Features
- Readability extraction — kills navbars, sidebars, ads, cookie banners. Falls back to cleaned
<body> when Readability returns too little (e.g. HN's table layout).
- Token budgeting —
--max-tokens N keeps all headings, fills remaining budget in document order, appends [truncated — N more tokens]. Uses 1 token ≈ 4 chars heuristic.
- Post-processing — strips HTML comments, zero-width chars, social CTAs, breadcrumbs, empty headings, collapses excess blank lines.
- Error handling — bad URLs, timeouts (15s), non-HTML content, missing files all exit code 1 with descriptive stderr.
- Output modes — plain markdown or
--json for programmatic use.
When to Use vs web_fetch
Use html2md when |
Use web_fetch when |
| Reading pages in cron jobs / sub-agents |
Quick one-off fetch in main session |
Token budget matters (--max-tokens) |
Page is a JSON/XML API endpoint |
| Heavy nav/ads/footers to strip |
JS rendering not needed |
| Need JSON output |
Simple pages |
Security Considerations
html2md fetches URLs and reads local files — that's its job. If you're passing untrusted input:
- URL fetching: the tool will fetch whatever URL it's given. Don't pass user-controlled URLs without validation if your threat model includes SSRF.
- File reading:
--file reads any path the process can access. In agent workflows, the agent controls the path — this is equivalent to the agent using cat.
- No shell execution: the tool itself never spawns shells or runs commands. When calling from scripts, use
execFileSync (not execSync) to avoid shell injection.
- No data exfiltration: output goes to stdout only. No network requests beyond the single URL fetch. No telemetry, no analytics, no phone-home.
- Dependencies: jsdom (Mozilla DOM implementation), Readability (Mozilla content extractor), Turndown (HTML→markdown). All widely audited, open source libraries.
Examples
# Read a Paul Graham essay within 2000 tokens
html2md --max-tokens 2000 https://paulgraham.com/greatwork.html
# HN front page as clean text, no link noise
html2md --no-links --no-images https://news.ycombinator.com
# Get token count before committing
html2md --json https://example.com | jq .tokens
# Pipe to file
html2md https://docs.example.com/api > api-docs.md
1---2name: html2md3description: Convert HTML pages to clean, agent-friendly markdown using Readability + Turndown. Strips navigation, ads, footers, cookie banners, social CTAs. Supports URL fetch, local files, stdin, token budgeting, and output flags. Ideal for research tasks, content extraction, and web scraping in agent workflows.4---5
6# html2md
7
8Aggressive HTML-to-markdown converter for AI agents. Mozilla Readability isolates main content, Turndown converts to markdown, then heavy post-processing strips remaining noise.
9
10> Full flag reference and advanced examples: `references/usage.md`
11
12## Setup
13
14```bash
15cd <skill-dir>/scripts
16npm install
17npm link # makes `html2md` globally available
18```
19
20Requires Node.js 22+.
21
22## Quick Start
23
24```bash
25html2md https://example.com # fetch + convert
26html2md --file page.html # local HTML file
27cat page.html | html2md --stdin # pipe from stdin
28html2md --max-tokens 2000 https://example.com # budget-aware truncation
29html2md --no-links https://example.com # strip hrefs, keep text
30html2md --json https://example.com # JSON: {title, url, markdown, tokens}
31```
32
33## Key Features
34
35- **Readability extraction** — kills navbars, sidebars, ads, cookie banners. Falls back to cleaned `<body>` when Readability returns too little (e.g. HN's table layout).
36- **Token budgeting** — `--max-tokens N` keeps all headings, fills remaining budget in document order, appends `[truncated — N more tokens]`. Uses 1 token ≈ 4 chars heuristic.
37- **Post-processing** — strips HTML comments, zero-width chars, social CTAs, breadcrumbs, empty headings, collapses excess blank lines.
38- **Error handling** — bad URLs, timeouts (15s), non-HTML content, missing files all exit code 1 with descriptive stderr.
39- **Output modes** — plain markdown or `--json` for programmatic use.
40
41## When to Use vs `web_fetch`
42
43| Use `html2md` when | Use `web_fetch` when |
44|-------------------|---------------------|
45| Reading pages in cron jobs / sub-agents | Quick one-off fetch in main session |
46| Token budget matters (`--max-tokens`) | Page is a JSON/XML API endpoint |
47| Heavy nav/ads/footers to strip | JS rendering not needed |
48| Need JSON output | Simple pages |
49
50## Security Considerations
51
52html2md fetches URLs and reads local files — that's its job. If you're passing untrusted input:
53
54- **URL fetching**: the tool will fetch whatever URL it's given. Don't pass user-controlled URLs without validation if your threat model includes SSRF.
55- **File reading**: `--file` reads any path the process can access. In agent workflows, the agent controls the path — this is equivalent to the agent using `cat`.
56- **No shell execution**: the tool itself never spawns shells or runs commands. When calling from scripts, use `execFileSync` (not `execSync`) to avoid shell injection.
57- **No data exfiltration**: output goes to stdout only. No network requests beyond the single URL fetch. No telemetry, no analytics, no phone-home.
58- **Dependencies**: jsdom (Mozilla DOM implementation), Readability (Mozilla content extractor), Turndown (HTML→markdown). All widely audited, open source libraries.
59
60## Examples
61
62```bash
63# Read a Paul Graham essay within 2000 tokens
64html2md --max-tokens 2000 https://paulgraham.com/greatwork.html
65
66# HN front page as clean text, no link noise
67html2md --no-links --no-images https://news.ycombinator.com
68
69# Get token count before committing
70html2md --json https://example.com | jq .tokens
71
72# Pipe to file
73html2md https://docs.example.com/api > api-docs.md
74```
modbender