tomevault-io GhostFetch
GhostFetch is a stealthy headless browser service designed specifically for AI agents. It bypasses anti-bot protections (like those on X/Twitter) and returns clean, LLM-friendly Markdown.
Installation
GhostFetch is already in your workspace at the repo root. To use it as a skill, ensure the Python environment is set up from the repo directory:
python3 -m venv venv
source venv/bin/activate
pip install -r requirements.txt
playwright install chromium
Usage
You can use GhostFetch in two ways:
1. Direct CLI (Recommended for quick fetches)
Use the scraper module directly to get Markdown from a URL.
python3 -m src.core.scraper "https://x.com/user/status/123"
2. API Mode (Background service)
Start the FastAPI server to handle concurrent jobs and webhooks.
# Start server
python3 main.py
# Fetch via API
curl -X POST "http://localhost:8000/fetch" \
-H "Content-Type: application/json" \
-d '{"url": "https://x.com/user/status/123"}'
Stealth Features
- Ghost Protocol: Advanced proxy rotation and hardware fingerprinting.
- X.com Optimized: Specific logic to wait for dynamic tweet content.
- LLM Ready: Automatic HTML-to-Markdown conversion.
Created by Syed Arsalan Shah (@iArsalanshah)
Converted and distributed by TomeVault — claim your Tome and manage your conversions.