Flyio Rate Limits

Fly.io Rate Limits

Overview

The Fly.io Machines API rate-limits per organization, with write operations (create, delete, update) throttled much more aggressively than reads. Deploying fleets of edge machines across multiple regions can easily trigger 429s, especially during rolling deployments or auto-scaling events. The API returns a Retry-After header on rate-limited responses, and organizations running 50+ machines should implement client-side token bucket limiting to avoid cascading failures during high-churn operations.

Prerequisites

• Current platform limits confirmed for the organization plus approved concurrency, retry bounds, and a queue owner.
• Redacted telemetry for request category, queue age, throttle count, and machine lifecycle—not tokens or payloads.
• A staging fleet/synthetic workload for testing pauses, retries, and cancellation.

Instructions

1. Honor explicit throttling guidance and use bounded concurrency with jittered backoff.
2. Attach idempotency/operation tracking to lifecycle changes so a retry cannot duplicate a create, stop, or delete action.
3. Queue exhausted operations for reviewed handling, alert on backlog growth, and reduce demand before resuming.

Output

Publish a rate-control receipt with policy version, concurrency, retry bounds, throttle count, queue outcome, owner, and manual disposition. Do not expose app names if they are sensitive, tokens, or request bodies.

Examples

Apply a small synthetic scale change, simulate a 429, and confirm the worker waits and then performs the operation once. A repeated failure must enter the review queue rather than trigger a fleet-wide replay.

Rate Limit Reference

Rate Limiter Implementation

class FlyRateLimiter {
  private tokens: number;
  private lastRefill: number;
  private readonly max: number;
  private readonly refillRate: number;
  private queue: Array<{ resolve: () => void }> = [];

  constructor(maxPerMinute: number) {
    this.max = maxPerMinute;
    this.tokens = maxPerMinute;
    this.lastRefill = Date.now();
    this.refillRate = maxPerMinute / 60_000;
  }

  async acquire(): Promise<void> {
    this.refill();
    if (this.tokens >= 1) { this.tokens -= 1; return; }
    return new Promise(resolve => this.queue.push({ resolve }));
  }

  private refill() {
    const now = Date.now();
    this.tokens = Math.min(this.max, this.tokens + (now - this.lastRefill) * this.refillRate);
    this.lastRefill = now;
    while (this.tokens >= 1 && this.queue.length) {
      this.tokens -= 1;
      this.queue.shift()!.resolve();
    }
  }
}

const writeLimiter = new FlyRateLimiter(8);  // leave headroom under 10/min
const readLimiter = new FlyRateLimiter(100);

Retry Strategy

async function flyRetry<T>(fn: () => Promise<Response>, maxRetries = 4): Promise<T> {
  for (let attempt = 0; attempt <= maxRetries; attempt++) {
    const res = await fn();
    if (res.ok) return res.json();
    if (res.status === 429) {
      const retryAfter = parseInt(res.headers.get("Retry-After") || "10", 10);
      const delay = retryAfter * 1000 + Math.random() * 2000;
      await new Promise(r => setTimeout(r, delay));
      continue;
    }
    if (res.status >= 500 && attempt < maxRetries) {
      await new Promise(r => setTimeout(r, Math.pow(2, attempt) * 1000));
      continue;
    }
    throw new Error(`Fly API ${res.status}: ${await res.text()}`);
  }
  throw new Error("Max retries exceeded");
}

Batch Processing

async function rollingDeployMachines(appId: string, configs: any[], batchSize = 3) {
  const results: any[] = [];
  for (let i = 0; i < configs.length; i += batchSize) {
    const batch = configs.slice(i, i + batchSize);
    const batchResults = await Promise.all(
      batch.map(async cfg => {
        await writeLimiter.acquire();
        return flyRetry(() =>
          fetch(`https://api.machines.dev/v1/apps/${appId}/machines`, {
            method: "POST", headers, body: JSON.stringify(cfg),
          })
        );
      })
    );
    results.push(...batchResults);
    if (i + batchSize < configs.length) await new Promise(r => setTimeout(r, 10_000));
  }
  return results;
}

Error Handling

Resources

• Fly.io Machines API

Next Steps

See flyio-performance-tuning.