# Data Harvester V2

> Batch web scraping for competitor analysis, price monitoring and market research

- Skill: `modbender/data-harvester-v2` (Agent Skill, multi-file: 2 files)
- Install (CLI): `npx skillmds@latest add modbender/data-harvester-v2`
- Raw SKILL.md: https://api.skillmd.com/api/skills/modbender/data-harvester-v2/raw
- Safety review: pending (external: skill-scanner PASS, skillspector PASS)
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: Research & Search
- Author: modbender (https://skillmd.com/u/modbender)
- Updated: 2026-09-09
- Page: https://skillmd.com/skills/modbender/data-harvester-v2

---


# 🧑‍🌾 Data Harvester - Batch Scraping & Competitor Analysis

Professional-grade batch web scraping tool for competitor monitoring, price tracking, and market research.

## Features

- **Batch URL Scraping**: Process multiple URLs from file
- **Competitor Comparison**: Compare products across platforms
- **Stock Data**: Real-time stock quotes and analysis
- **Sector Trends**: Hot sectors and market trends
- **News Aggregation**: Collect news by keywords

## Commands

| Command | Description | Example |
|---------|-------------|---------|
| `stock <code>` | Query stock data | `/data-harvester stock 600519` |
| `block` | Hot market sectors | `/data-harvester block` |
| `fund` | Money flow tracking | `/data-harvester fund` |
| `news <keyword>` | News search | `/data-harvester news AI` |
| `compare <product>` | Competitor comparison | `/data-harvester compare laptop` |
| `batch <file>` | Batch URL list | `/data-harvester batch urls.txt` |
| `export <file> <format>` | Export data | `/data-harvester export data.json excel` |

## Usage Examples

### Stock Query
```
/data-harvester stock 000001
```
Returns: Real-time stock price, change %, volume

### Competitor Comparison
```
/data-harvester compare "wireless earphones"
```
Returns: Price comparison across Taobao, JD, Pinduoduo

### Sector Analysis
```
/data-harvester block
```
Returns: Top 7hot sectors with leader stocks

### Fund Flow
```
/data-harvester fund
```
Returns: Top 5 stocks with main fund inflow

## Batch Processing

Create a text file with one URL per line:
```
https://example1.com
https://example2.com
https://example3.com
```

Then run:
```
/data-harvester batch urls.txt
```

## Technical Details

- **Language**: Python 3
- **Output Formats**: JSON, CSV, Excel
- **Rate Limiting**: Configurable delay between requests

## Legal Notice

- Respect target website's robots.txt
- Use reasonable request intervals (3-5 seconds)
- For educational/research purposes only
- Commercial use requires proper authorization