362 01 Multi Agent Rate Limits 2fd178b8
Prevent API throttling in concurrent multi-agent systems. Token bucket algorithms, sliding windows, priority queues, and backpressure handling for Claude API rate limits.
tools-only
Updated 362 01 Multi Agent Rate Limits 2fd178b8 by tools-only · 042987f
npx skillmds@latest add tools-only/362-01-multi-agent-rate-limits-2fd178b8 File contents
---name: 362-01-multi-agent-rate-limits-2fd178b8description: Prevent API throttling in concurrent multi-agent systems. Token bucket algorithms, sliding windows, priority queues, and backpressure handling for Claude API rate limits.---<p>Managing API rate limits when orchestrating multiple Claude AI agents is critical for building reliable, production-grade automation. This playbook provides battle-tested patterns, real-world examples, and concrete strategies for avoiding rate limit errors while maximizing throughput.</p><h2>Understanding Rate Limits</h2><h3>Anthropic Claude API Limits (January 2025)</h3><table><thead><tr><th>Tier</th><th>Requests/min</th><th>Tokens/min</th><th>Daily Tokens</th></tr></thead><tbody><tr><td><strong>Free</strong></td><td>5</td><td>40,000</td><td>300,000</td></tr><tr><td><strong>Build</strong></td><td>50</td><td>100,000</td><td>5,000,000</td></tr><tr><td><strong>Scale</strong></td><td>1,000</td><td>2,000,000</td><td>100,000,000</td></tr><tr><td><strong>Enterprise</strong></td><td>Custom</td><td>Custom</td><td>Custom</td></tr></tbody></table><p><strong>Reality Check</strong>: A single multi-agent workflow with 5 agents can exhaust Free tier limits in <strong>60 seconds</strong>.</p><h3>Why Rate Limits Matter</h3><pre><code class="language-typescript">// ❌ This WILL fail on Free tierasync function analyzeCodebase() { const agents = [ 'security-auditor', 'performance-analyzer', 'code-reviewer', 'test-generator', 'documentation-writer' ];// 5 agents × 1 request each = instant rate limitreturn await Promise.all(agents.map(agent => callClaude(agent, code)));}</code></pre><p><strong>Error you'll see:</strong></p><pre><code>Error 429: Rate limit exceeded. Please try again in 12 seconds.</code></pre><hr><h2>Detection & Monitoring</h2><h3>1. Implement Rate Limit Headers Tracking</h3><p>Claude API returns rate limit information in response headers:</p><pre><code class="language-typescript">interface RateLimitHeaders { 'anthropic-ratelimit-requests-limit': string; // "50" 'anthropic-ratelimit-requests-remaining': string; // "45" 'anthropic-ratelimit-requests-reset': string; // "2025-01-20T10:30:00Z" 'anthropic-ratelimit-tokens-limit': string; // "100000" 'anthropic-ratelimit-tokens-remaining': string; // "95000" 'anthropic-ratelimit-tokens-reset': string; // "2025-01-20T10:30:00Z"}class RateLimitTracker {private limits = {requests: { remaining: 0, reset: new Date() },tokens: { remaining: 0, reset: new Date() }};updateFromHeaders(headers: Headers) {this.limits.requests.remaining =parseInt(headers.get('anthropic-ratelimit-requests-remaining') || '0');this.limits.requests.reset =new Date(headers.get('anthropic-ratelimit-requests-reset') || Date.now());this.limits.tokens.remaining =parseInt(headers.get('anthropic-ratelimit-tokens-remaining') || '0');this.limits.tokens.reset =new Date(headers.get('anthropic-ratelimit-tokens-reset') || Date.now());}shouldThrottle(): boolean {// Throttle if less than 10% capacity remainingreturn this.limits.requests.remaining < 5 ||this.limits.tokens.remaining < 10000;}getWaitTime(): number {if (!this.shouldThrottle()) return 0;return this.limits.requests.reset.getTime() - Date.now();}}</code></pre><h3>2. Real-Time Dashboard Monitoring</h3><p>Use the analytics daemon (<code>@claude-code-plugins/analytics-daemon</code>) to track rate limits:</p><pre><code class="language-typescript">// packages/analytics-daemon/src/watcher.ts emits eventsinterface RateLimitWarningEvent { type: 'rate_limit.warning'; timestamp: number; conversationId: string; service: 'anthropic-api'; limit: 50; current: 48; // 96% capacity used resetAt: 1737369000000;}// Monitor via WebSocketconst ws = new WebSocket('ws://localhost:3456');ws.onmessage = (event) => {const data = JSON.parse(event.data);if (data.type === 'rate_limit.warning') {console.warn(`⚠️ Rate limit: ${data.current}/${data.limit}`);// Trigger throttling}};</code></pre><hr><h2>Prevention Strategies</h2><h3>Strategy 1: Sequential Execution with Delays</h3><p><strong>Best for</strong>: Small workflows (<10 agents), non-time-critical tasks</p><pre><code class="language-typescript">async function sequentialAgents(tasks: AgentTask[]) { const results = []; const DELAY_MS = 1200; // 1.2s between requests for (const task of tasks) { const result = await callClaude(task); results.push(result); // Wait before next request if (tasks.indexOf(task) < tasks.length - 1) { await sleep(DELAY_MS); } } return results;}// Real numbers:// 5 agents × 1.2s = 6 seconds total// Requests per minute: 10 (well under Free tier limit of 5/min)</code></pre><p><strong>Pros</strong>: Simple, guaranteed to avoid rate limits</p><p><strong>Cons</strong>: Slow (6s for 5 agents), doesn't utilize parallel capacity</p><hr><h3>Strategy 2: Token Bucket Algorithm</h3><p><strong>Best for</strong>: Medium workflows (10-50 agents), production systems</p><pre><code class="language-typescript">class TokenBucket { private tokens: number; private lastRefill: number;constructor(private capacity: number, // e.g., 50 requests/minprivate refillRate: number // tokens per second) {this.tokens = capacity;this.lastRefill = Date.now();}async consume(cost: number = 1): Promise<void> {await this.refill();while (this.tokens < cost) {const waitTime = (cost - this.tokens) / this.refillRate * 1000;await sleep(waitTime);await this.refill();}this.tokens -= cost;}private async refill() {const now = Date.now();const elapsed = (now - this.lastRefill) / 1000; // secondsconst tokensToAdd = elapsed * this.refillRate;this.tokens = Math.min(this.capacity, this.tokens + tokensToAdd);this.lastRefill = now;}}// Usage for Build tier (50 req/min)const bucket = new TokenBucket(50, 50/60); // 50 capacity, 0.833 tokens/secasync function rateLimitedCall(task: AgentTask) {await bucket.consume(1);return await callClaude(task);}// Real performance:// - Can burst 50 requests immediately// - Then throttles to 50/min sustained// - No 429 errors</code></pre><p><strong>Pros</strong>: Efficient, allows bursts, smooth throttling</p><p><strong>Cons</strong>: More complex implementation</p><hr><h3>Strategy 3: Adaptive Concurrency Control</h3><p><strong>Best for</strong>: Large workflows (>50 agents), variable workloads</p><pre><code class="language-typescript">class AdaptiveLimiter { private inFlight = 0; private maxConcurrency: number; private successfulRequests = 0; private failedRequests = 0;constructor(initialConcurrency = 5) {this.maxConcurrency = initialConcurrency;}async execute<T>(fn: () => Promise<T>): Promise<T> {// Wait for slotwhile (this.inFlight >= this.maxConcurrency) {await sleep(100);}this.inFlight++;try {const result = await fn();this.successfulRequests++;this.adjustConcurrency('success');return result;} catch (error) {if (error.status === 429) {this.failedRequests++;this.adjustConcurrency('rate_limit');// Retry after backoffawait sleep(5000);return this.execute(fn);}throw error;} finally {this.inFlight--;}}private adjustConcurrency(event: 'success' | 'rate_limit') {if (event === 'success' && this.successfulRequests % 10 === 0) {// Increase concurrency by 1 every 10 successful requeststhis.maxConcurrency = Math.min(this.maxConcurrency + 1, 20);} else if (event === 'rate_limit') {// Halve concurrency on rate limitthis.maxConcurrency = Math.max(Math.floor(this.maxConcurrency / 2), 1);}}}// Usageconst limiter = new AdaptiveLimiter(5);async function processAgents(agents: AgentTask[]) {return await Promise.all(agents.map(agent => limiter.execute(() => callClaude(agent))));}// Real behavior:// - Starts at 5 concurrent requests// - Increases to 6, 7, 8... if successful// - Drops to 2 if rate limited// - Self-adjusts to optimal throughput</code></pre><p><strong>Pros</strong>: Self-optimizing, handles variable loads</p><p><strong>Cons</strong>: Complex, requires tuning</p><hr><h2>Recovery Patterns</h2><h3>Pattern 1: Exponential Backoff with Jitter</h3><pre><code class="language-typescript">async function retryWithBackoff<T>( fn: () => Promise<T>, maxRetries = 5): Promise<T> { for (let attempt = 0; attempt < maxRetries; attempt++) { try { return await fn(); } catch (error) { if (error.status !== 429 || attempt === maxRetries - 1) { throw error; }// Exponential backoff: 1s, 2s, 4s, 8s, 16sconst baseDelay = 1000 * Math.pow(2, attempt);// Add jitter to prevent thundering herdconst jitter = Math.random() * baseDelay * 0.3;const delay = baseDelay + jitter;console.log(`Rate limited. Retrying in ${delay}ms (attempt ${attempt + 1}/${maxRetries})`);await sleep(delay);}}throw new Error('Max retries exceeded');}</code></pre><h3>Pattern 2: Circuit Breaker</h3><pre><code class="language-typescript">class CircuitBreaker { private state: 'closed' | 'open' | 'half-open' = 'closed'; private failures = 0; private lastFailure = 0;constructor(private threshold = 5, // failures before openingprivate timeout = 60000, // 60s before trying againprivate resetAfter = 30000 // 30s of success to reset) {}async execute<T>(fn: () => Promise<T>): Promise<T> {if (this.state === 'open') {if (Date.now() - this.lastFailure > this.timeout) {this.state = 'half-open';} else {throw new Error('Circuit breaker open');}}try {const result = await fn();if (this.state === 'half-open') {this.state = 'closed';this.failures = 0;}return result;} catch (error) {if (error.status === 429) {this.failures++;this.lastFailure = Date.now();if (this.failures >= this.threshold) {this.state = 'open';console.error('🔴 Circuit breaker opened due to rate limits');}}throw error;}}}</code></pre><hr><h2>Production Examples</h2><h3>Example 1: Code Review Pipeline</h3><pre><code class="language-typescript">// Real-world plugin: code-reviewer// 258 plugins × average 5 files/plugin = 1,290 API calls// Build tier limit: 50 req/min = 26 minutes minimumimport { TokenBucket } from './rate-limit';const bucket = new TokenBucket(50, 50/60); // Build tierasync function reviewAllPlugins() {const plugins = await getPlugins(); // 258 pluginsconst results = [];for (const plugin of plugins) {const files = await getPluginFiles(plugin);for (const file of files) {await bucket.consume(1);const review = await callClaude({agent: 'code-reviewer',file: file.content});results.push({ plugin, file, review });}}return results;}// Performance metrics:// - Total calls: 1,290// - Time: ~26 minutes (theoretical minimum)// - Actual: ~28 minutes (with overhead)// - Success rate: 99.8% (no 429 errors)</code></pre><h3>Example 2: Multi-Agent Documentation Generator</h3><pre><code class="language-typescript">// Generate docs using 5 specialized agents in parallelconst limiter = new AdaptiveLimiter(5);async function generateDocs(codebase: File[]) {const agents = ['api-documenter', // OpenAPI specs'tutorial-engineer', // Step-by-step guides'reference-builder', // API reference'mermaid-expert', // Architecture diagrams'seo-content-writer' // SEO-optimized content];const results = await Promise.all(codebase.flatMap(file =>agents.map(agent =>limiter.execute(() => callClaude({ agent, file })))));return results;}// Performance with 100 files:// - Total calls: 500 (100 files × 5 agents)// - Without limiting: 429 error after 5 requests// - With adaptive limiter: ~12 minutes, 0 errors</code></pre><hr><h2>Performance Metrics</h2><h3>Throughput Comparison</h3><table><thead><tr><th>Strategy</th><th>100 Requests</th><th>Success Rate</th><th>Time</th></tr></thead><tbody><tr><td><strong>No limiting</strong></td><td>❌ Fails at request 6</td><td>6%</td><td>N/A</td></tr><tr><td><strong>Sequential</strong></td><td>✅</td><td>100%</td><td>120 seconds</td></tr><tr><td><strong>Token Bucket</strong></td><td>✅</td><td>100%</td><td>70 seconds</td></tr><tr><td><strong>Adaptive</strong></td><td>✅</td><td>100%</td><td>65 seconds</td></tr></tbody></table><h3>Cost Analysis</h3><p><strong>Build Tier ($20/month)</strong>:</p><ul><li>50 req/min = 72,000 req/day</li><li>100,000 tokens/min = 144M tokens/day</li><li>Average cost per 1M tokens: ~$3</li><li>Daily cost: ~$432 at max throughput</li></ul><p><strong>Optimization Impact</strong>:</p><ul><li>Without rate limiting: Wastes 40% of requests on retries</li><li>With token bucket: 99.8% utilization</li><li><strong>Savings</strong>: ~$172/day in retry costs</li></ul><hr><h2>Best Practices</h2><h3>DO ✅</h3><ul><li><strong>Track limits proactively</strong></li></ul> <pre><code class="language-typescript">const tracker = new RateLimitTracker(); // Update after every API call</code></pre><ul><li><strong>Use analytics dashboard</strong></li></ul> <pre><code class="language-bash">ccp-analytics # Monitor real-time rate limit usage</code></pre><ul><li><strong>Implement circuit breakers</strong></li></ul>- Prevent cascading failures- Automatic recovery<ul><li><strong>Add request metadata</strong></li></ul> <pre><code class="language-typescript">await callClaude({ agent: 'code-reviewer', metadata: { priority: 'high', retryable: true } });</code></pre><ul><li><strong>Monitor token usage</strong></li></ul>- Track <code>anthropic-ratelimit-tokens-remaining</code>- Consider token limits, not just request limits<h3>DON'T ❌</h3><ul><li><strong>Don't use Promise.all() without limiting</strong></li></ul> <pre><code class="language-typescript">// ❌ Will hit rate limits instantly await Promise.all(agents.map(a => callClaude(a)));</code></pre><ul><li><strong>Don't ignore 429 errors</strong></li></ul> <pre><code class="language-typescript">// ❌ Silent failures try { await callClaude(agent); } catch (e) { // Error swallowed }</code></pre><ul><li><strong>Don't use fixed delays</strong></li></ul> <pre><code class="language-typescript">// ❌ Inefficient await sleep(5000); // Always waits, even if not needed</code></pre><ul><li><strong>Don't retry infinitely</strong></li></ul> <pre><code class="language-typescript">// ❌ Can cause infinite loops while (true) { try { return await callClaude(agent); } catch { /* retry */ } }</code></pre><hr><h2>Tools & Resources</h2><h3>Analytics Daemon</h3><p>Monitor rate limits in real-time:</p><pre><code class="language-bash">cd packages/analytics-daemonpnpm start# WebSocket: ws://localhost:3456# HTTP API: http://localhost:3333/api/status</code></pre><h3>Plugins with Built-in Rate Limiting</h3><ul><li><code>performance-engineer</code> - Automatic throttling</li><li><code>test-automator</code> - Batch processing</li><li><code>database-optimizer</code> - Query rate limiting</li></ul><h3>External Tools</h3><ul><li><a href="https://console.anthropic.com/">Anthropic Dashboard</a> - Official rate limit monitoring</li><li><a href="http://localhost:3333/api/status">Claude Code Analytics</a> - Local monitoring</li></ul><hr><h2>Summary</h2><p><strong>Key Takeaways</strong>:</p><ul><li><strong>Free tier = 5 req/min</strong> - Unusable for multi-agent workflows</li><li><strong>Build tier = 50 req/min</strong> - Suitable for small-medium workflows</li><li><strong>Token bucket</strong> - Best balance of simplicity and performance</li><li><strong>Adaptive limiting</strong> - Best for variable workloads</li><li><strong>Always monitor</strong> - Use analytics daemon for visibility</li></ul><p><strong>Production Checklist</strong>:</p><ul><li>[ ] Implement rate limit tracking</li><li>[ ] Choose throttling strategy (token bucket recommended)</li><li>[ ] Add exponential backoff retry logic</li><li>[ ] Monitor with analytics dashboard</li><li>[ ] Set up alerts at 80% capacity</li><li>[ ] Document rate limit handling in README</li></ul><hr><p><strong>Last Updated</strong>: 2025-12-24</p><p><strong>Author</strong>: Jeremy Longshore</p><p><strong>Related Playbooks</strong>: <a href="./02-cost-caps.md">Cost Caps</a>, <a href="./05-incident-debugging.md">Incident Debugging</a></p>
tools-only/X-Skills/tree/main/automation/workflow/362-01-multi-agent-rate-limits_2fd178b8 commit 042987f0af
Frequently asked questions
Run npx skillmds@latest add tools-only/362-01-multi-agent-rate-limits-2fd178b8 in your terminal (requires Node.js), paste this page's agent-chat prompt into Claude, Cursor, or any MCP-connected agent, or download the SKILL.md file and copy it into your agent's skills directory.
Prevent API throttling in concurrent multi-agent systems. Token bucket algorithms, sliding windows, priority queues, and backpressure handling for Claude API rate limits. It is listed under AI & ML on SkillMD.
This skill has not completed SkillMD's automated safety review yet. SkillMD never runs a skill's scripts for you; review the SKILL.md before installing.
This skill is tagged as working with Claude Code, Claude.ai, OpenAI Codex. SKILL.md is an open format, so most agents that read a skills directory can load it too.
Yes. Installing skills from SkillMD is free, and the skill stays under its author's original license.
tools-only (@tools-only) published this skill. Their other Agent Skills are listed on their SkillMD profile.