# 2032 Ard Cbc34d45 > ARD: Aggregating Crypto News - Skill: `tools-only/2032-ard-cbc34d45` (Agent Skill, multi-file: 3 files) - Install (CLI): `npx skillmds@latest add tools-only/2032-ard-cbc34d45` - Raw SKILL.md: https://api.skillmd.com/api/skills/tools-only/2032-ard-cbc34d45/raw - Safety review: pending (external: skill-scanner WARNING, skillspector PASS) - Works with: Claude Code, Claude.ai, OpenAI Codex - Category: Research & Search - Author: tools-only (https://skillmd.com/u/tools-only) - Updated: 2026-09-22 - Page: https://skillmd.com/skills/tools-only/2032-ard-cbc34d45 --- # ARD: Aggregating Crypto News ## Document Control | Field | Value | |-------|-------| | **Skill Name** | aggregating-crypto-news | | **Architecture Pattern** | Data Aggregation Pipeline | | **Version** | 2.0.0 | | **Author** | Jeremy Longshore | --- ## Architectural Overview ### Pattern: Multi-Source Data Aggregation This skill implements a parallel fetching, aggregation, and filtering pipeline for RSS-based news sources. ``` ┌─────────────────────────────────────────────────────────────────────────────┐ │ CRYPTO NEWS AGGREGATOR ARCHITECTURE │ └─────────────────────────────────────────────────────────────────────────────┘ ┌──────────────────┐ ┌──────────────────┐ ┌──────────────────┐ │ RSS Source 1 │ │ RSS Source 2 │ │ RSS Source N │ │ (CoinDesk) │ │ (CoinTelegraph) │ │ (The Block) │ └────────┬─────────┘ └────────┬─────────┘ └────────┬─────────┘ │ │ │ └────────────────────────┼────────────────────────┘ │ ▼ ┌─────────────────────────┐ │ FEED FETCHER │ │ (Parallel Requests) │ │ - Timeout handling │ │ - Caching layer │ └────────────┬────────────┘ │ ▼ ┌─────────────────────────┐ │ FEED PARSER │ │ - XML/RSS parsing │ │ - Date normalization │ │ - Content extraction │ └────────────┬────────────┘ │ ▼ ┌─────────────────────────┐ │ DEDUPLICATOR │ │ - Title similarity │ │ - URL matching │ └────────────┬────────────┘ │ ▼ ┌─────────────────────────┐ │ SCORER │ │ - Keyword matching │ │ - Market-moving boost │ │ - Source quality │ └────────────┬────────────┘ │ ▼ ┌─────────────────────────┐ │ FILTER ENGINE │ │ - Time window │ │ - Coin matching │ │ - Category filtering │ └────────────┬────────────┘ │ ▼ ┌─────────────────────────┐ │ FORMATTER │ │ - Table output │ │ - JSON export │ │ - CSV export │ └─────────────────────────┘ ``` ### Workflow 1. **Fetch**: Parallel HTTP requests to RSS feeds with timeout handling 2. **Parse**: Extract title, content, date, source from XML 3. **Dedupe**: Remove duplicate articles based on title similarity 4. **Score**: Calculate relevance score based on keywords and source 5. **Filter**: Apply user filters (time, coin, category) 6. **Format**: Output in requested format --- ## Progressive Disclosure Strategy ### Level 1: Simple Scan (Default) ```bash python news_aggregator.py ``` Returns top 20 news from past 24h, relevance sorted. ### Level 2: Filtered Scan ```bash python news_aggregator.py --coin BTC --period 4h --top 10 ``` Adds coin and time filtering. ### Level 3: Category + Export ```bash python news_aggregator.py --category defi --format json --output news.json ``` Full filtering with export. ### Level 4: Advanced Configuration ```bash python news_aggregator.py --sources coindesk,theblock --min-score 50 --verbose ``` Source selection and score thresholds. --- ## Tool Permission Strategy ### Allowed Tools (Scoped) ```yaml allowed-tools: Read, Bash(crypto:news-*) ``` | Tool | Scope | Purpose | |------|-------|---------| | Read | Unrestricted | Read config, sources list | | Bash | `crypto:news-*` | Execute news aggregation scripts | ### Why These Tools - **Read**: Load source configuration and settings - **Bash(crypto:news-*)**: Execute Python scripts for fetching and parsing - **No Write**: Aggregation is read-only; exports use script file output --- ## Directory Structure ``` plugins/crypto/crypto-news-aggregator/ └── skills/ └── aggregating-crypto-news/ ├── PRD.md # Product requirements ├── ARD.md # This file ├── SKILL.md # Core instructions ├── scripts/ │ ├── news_aggregator.py # Main CLI entry point │ ├── feed_fetcher.py # Parallel RSS fetching │ ├── feed_parser.py # XML parsing and normalization │ ├── scorer.py # Relevance scoring │ └── formatters.py # Output formatting ├── references/ │ ├── errors.md # Error handling guide │ └── examples.md # Usage examples └── config/ ├── settings.yaml # Configuration options └── sources.yaml # RSS source registry ``` --- ## Data Flow Architecture ### Input - User request with optional filters (--coin, --period, --category) - RSS feed URLs from source registry ### Processing Pipeline ``` Input Request │ ├──► Load sources.yaml │ │ │ ▼ ├──► Parallel fetch (ThreadPoolExecutor) │ │ │ ▼ ├──► Parse each feed (feedparser library) │ │ │ ▼ ├──► Normalize entries to common schema: │ { │ "title": str, │ "url": str, │ "source": str, │ "published": datetime, │ "summary": str, │ "category": str, │ "relevance_score": float │ } │ │ │ ▼ ├──► Deduplicate by title similarity (>80% match) │ │ │ ▼ ├──► Score relevance: │ - Base score from source quality (1-10) │ - Keyword boost (+10 per market-moving term) │ - Recency boost (newer = higher) │ │ │ ▼ ├──► Apply filters: │ - Time window (published > now - period) │ - Coin match (title/summary contains symbol) │ - Category match (source category or keywords) │ │ │ ▼ └──► Format and output ``` ### Output Schema ```json { "articles": [ { "rank": 1, "title": "Bitcoin Hits New All-Time High", "url": "https://coindesk.com/...", "source": "CoinDesk", "published": "2026-01-14T10:30:00Z", "age": "2h ago", "category": "market", "relevance_score": 85.5, "coins_mentioned": ["BTC"] } ], "meta": { "total_fetched": 500, "after_dedup": 380, "after_filter": 25, "sources_used": 12, "period": "24h", "filters": {"category": "market"} } } ``` --- ## Error Handling Strategy ### Error Categories | Category | Examples | Strategy | |----------|----------|----------| | Network | Timeout, DNS failure | Skip source, continue with others | | Parse | Malformed XML, missing fields | Skip entry, log warning | | Filter | Invalid coin symbol | Warn and proceed with partial filter | | Export | File permission denied | Error with clear message | ### Graceful Degradation ``` Full Success (all sources) │ ▼ Partial Success (some sources failed) │ → Log warnings, return available data │ ▼ Cached Fallback (all sources failed) │ → Return cached data with stale warning │ ▼ Complete Failure │ → Clear error message with remediation ``` --- ## Composability & Stacking ### As a Standalone Skill - Fully functional for news aggregation - No dependencies on other skills ### As Data Provider Skills that can consume news data: - **market-sentiment-analyzer**: Feed articles for sentiment scoring - **crypto-signal-generator**: Use news as signal input - **whale-alert-monitor**: Correlate whale activity with news ### Integration Pattern ```python # In another skill's script: from pathlib import Path import subprocess import json def get_news_for_coin(coin: str, period: str = "4h") -> list: """Fetch news using crypto-news-aggregator skill.""" news_script = Path(__file__).parent.parent.parent.parent / \ "crypto-news-aggregator/skills/aggregating-crypto-news/scripts/news_aggregator.py" result = subprocess.run( ["python3", str(news_script), "--coin", coin, "--period", period, "--format", "json"], capture_output=True, text=True ) if result.returncode == 0: return json.loads(result.stdout).get("articles", []) return [] ``` --- ## Performance & Scalability ### Performance Targets | Metric | Target | Approach | |--------|--------|----------| | Fetch time (50 sources) | < 10s | Parallel fetching with ThreadPoolExecutor | | Parse time (500 articles) | < 2s | Efficient XML parsing with feedparser | | Memory usage | < 100MB | Stream processing, no full corpus in memory | ### Optimization Strategies 1. **Parallel Fetching**: Use ThreadPoolExecutor(max_workers=10) 2. **Caching**: Cache responses with 5-minute TTL 3. **Lazy Parsing**: Parse only after filter candidates identified 4. **Early Exit**: Stop processing when --top limit reached ### Scalability Considerations - **More Sources**: Linear scaling with parallel fetch - **Higher Volume**: Pagination and streaming for large exports - **Frequency**: Caching prevents redundant fetches --- ## Testing Strategy ### Unit Tests | Component | Test Cases | |-----------|------------| | FeedFetcher | Timeout handling, cache hit/miss, parallel execution | | FeedParser | Valid RSS, malformed XML, missing fields | | Scorer | Keyword matching, source quality, recency boost | | Formatters | Table, JSON, CSV output validation | ### Integration Tests - End-to-end with mock RSS server - Real feeds (selected stable sources) - Export file validation ### Manual Testing ```bash # Default scan python news_aggregator.py # Coin filter python news_aggregator.py --coin BTC # Category filter python news_aggregator.py --category defi # Export python news_aggregator.py --format json --output test.json ``` --- ## Security & Compliance ### Security Considerations - **No Authentication**: RSS feeds are public, no credentials needed - **Input Validation**: Sanitize coin symbols and category names - **Output Sanitization**: Escape HTML in article titles/summaries ### Data Privacy - No user data collected or stored - No tracking or analytics - Cached data is ephemeral (5-minute TTL) ### Rate Limiting - Respect source rate limits (1 req/source/5min default) - No aggressive crawling - Caching reduces request volume