firecrawl-load-scale
Skill Scaffold
firecrawl-load-scale/
SKILL.md
File Descriptions
1. SKILL.md
Purpose: Implement load testing, horizontal scaling strategies, and capacity planning for high-volume scraping operations. Workflow: Performance engineering skill - ensures scraping infrastructure can handle target throughput. Relates to: Builds on firecrawl-batch-processing; informs firecrawl-architecture-variants
Summary
This skill implements performance engineering for high-volume FireCrawl operations. It covers load testing scraping pipelines (k6, Artillery, custom scripts), identifying bottlenecks (API limits, processing, storage), horizontal scaling patterns (worker pools, distributed processing), capacity planning for target throughput, and cost modeling at scale. These techniques ensure scraping infrastructure can reliably handle production data volumes.