# News Fetcher

> Install, configure, validate, and run the news-fetcher Python CLI for aggregating RSS/Atom and HTML news sources with deduplication, clustering, ranking, source diversity, and summaries. Use when an agent needs to fetch news, create or validate a config, troubleshoot news-fetcher installation, or produce JSON/Markdown/CSV/RSS output from multiple sources.

- Skill: `dvcrn/news-fetcher` (Agent Skill)
- Install (CLI): `npx skillmds@latest add dvcrn/news-fetcher`
- Raw SKILL.md: https://api.skillmd.com/api/skills/dvcrn/news-fetcher/raw
- Safety review: pending (external: skill-scanner PASS, skillspector PASS)
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Web & Frontend
- Author: dvcrn (https://skillmd.com/u/dvcrn)
- Updated: 2026-09-08
- Page: https://skillmd.com/skills/dvcrn/news-fetcher

---


# News Fetcher

Use this skill to get a working `news-fetcher` installation and run it correctly.

## Important

- Installing the ClawHub skill does **not** install the Python package.
- Install the Python package separately with `pip`.
- Put global options **before** `run`.

Correct:

```bash
news-fetcher --config config.yaml --limit 10 run
```

Wrong:

```bash
news-fetcher run --config config.yaml --limit 10
```

## Minimal working install

```bash
python3 -m venv .venv
. .venv/bin/activate
pip install --upgrade pip
pip install "git+https://github.com/miniade/news-fetcher.git@v0.1.4"
news-fetcher version
```

Expect `news-fetcher version 0.1.4`.

## Minimal working config

Generate a starter config:

```bash
news-fetcher config example > config.yaml
```

Or create one manually:

```yaml
sources:
  - name: BBC News
    url: http://feeds.bbci.co.uk/news/rss.xml
    weight: 1.0
    type: rss

  - name: Reuters Tech
    url: https://www.reutersagency.com/feed/?best-topics=tech
    weight: 1.2
    type: rss

  - name: Example HTML Source
    url: https://example.com/news
    weight: 0.9
    type: html
    selector: main article

thresholds:
  similarity: 0.8
  min_score: 0.3
  cluster_size: 2
  max_per_source: 3

weights:
  content: 0.6
  source: 0.2
  publish_time: 0.2
```

Validate it:

```bash
news-fetcher config validate config.yaml
```

## Common commands

Run with a config:

```bash
news-fetcher --config config.yaml --limit 20 run
```

Write Markdown output:

```bash
news-fetcher --config config.yaml --format markdown --output news.md run
```

Filter by time:

```bash
news-fetcher --config config.yaml --since 2026-03-01T00:00:00 run
```

Raise the score threshold:

```bash
news-fetcher --config config.yaml --min-score 0.5 run
```

Override sources directly from the CLI:

```bash
news-fetcher --sources "http://feeds.bbci.co.uk/news/rss.xml,https://news.ycombinator.com/rss" --limit 10 run
```

## HTML sources

For `type: html` sources, set `selector` when the site does not use clean `<article>` blocks.

Example:

```yaml
sources:
  - name: Example HTML Source
    url: https://example.com/news
    type: html
    selector: .story-card
```

## Troubleshooting

If `news-fetcher` is missing after installing the skill, install the Python package:

```bash
pip install "git+https://github.com/miniade/news-fetcher.git@v0.1.4"
```

If config validation fails, run:

```bash
news-fetcher config validate config.yaml
```

If results are too sparse, lower `thresholds.min_score` or raise `thresholds.max_per_source`.

## References

- Tagged source install: `pip install "git+https://github.com/miniade/news-fetcher.git@v0.1.4"`
- PyPI (optional, if published): https://pypi.org/project/news-fetcher/
- Source: https://github.com/miniade/news-fetcher
- Version targeted by this skill: 0.1.4