Hex Rate Limits

Hex Rate Limits

Overview

Hex's API enforces tight limits on project run triggers (20 per minute, 60 per hour) while leaving read operations like status checks and project listing largely unthrottled. Data teams scheduling batch analytics runs or triggering parameterized notebooks from CI/CD pipelines must carefully manage the hourly cap, since a single pipeline triggering 15 projects can consume a quarter of the hourly budget. Polling run status is free, but triggering runs is the bottleneck that shapes integration architecture.

Rate Limit Reference

Rate Limiter Implementation

class HexRateLimiter {
  private minuteTokens: number = 20;
  private hourlyTokens: number = 60;
  private lastMinuteRefill: number = Date.now();
  private lastHourlyRefill: number = Date.now();
  private queue: Array<{ resolve: () => void }> = [];

  async acquire(): Promise<void> {
    this.refill();
    if (this.minuteTokens >= 1 && this.hourlyTokens >= 1) {
      this.minuteTokens -= 1;
      this.hourlyTokens -= 1;
      return;
    }
    return new Promise(resolve => this.queue.push({ resolve }));
  }

  private refill() {
    const now = Date.now();
    this.minuteTokens = Math.min(20, this.minuteTokens + ((now - this.lastMinuteRefill) / 60_000) * 20);
    this.lastMinuteRefill = now;
    this.hourlyTokens = Math.min(60, this.hourlyTokens + ((now - this.lastHourlyRefill) / 3_600_000) * 60);
    this.lastHourlyRefill = now;
    while (this.minuteTokens >= 1 && this.hourlyTokens >= 1 && this.queue.length) {
      this.minuteTokens -= 1;
      this.hourlyTokens -= 1;
      this.queue.shift()!.resolve();
    }
  }
}

const runLimiter = new HexRateLimiter();

Retry Strategy

async function hexRunWithRetry(
  projectId: string, params: Record<string, any>, maxRetries = 3
): Promise<any> {
  for (let attempt = 0; attempt <= maxRetries; attempt++) {
    await runLimiter.acquire();
    const res = await fetch(`${HEX_BASE}/api/v1/run/${projectId}`, {
      method: "POST", headers,
      body: JSON.stringify({ inputParams: params }),
    });
    if (res.ok) return res.json();
    if (res.status === 429) {
      const delay = 30_000 * Math.pow(2, attempt) + Math.random() * 5000;
      await new Promise(r => setTimeout(r, delay));
      continue;
    }
    if (res.status >= 500 && attempt < maxRetries) {
      await new Promise(r => setTimeout(r, Math.pow(2, attempt) * 3000));
      continue;
    }
    throw new Error(`Hex API ${res.status}: ${await res.text()}`);
  }
  throw new Error("Max retries exceeded");
}

Batch Processing

async function batchRunProjects(projects: Array<{ id: string; params: any }>, batchSize = 5) {
  const results: any[] = [];
  for (let i = 0; i < projects.length; i += batchSize) {
    const batch = projects.slice(i, i + batchSize);
    const runs = await Promise.all(
      batch.map(p => hexRunWithRetry(p.id, p.params))
    );
    // Poll for completion
    for (const run of runs) {
      let status = run;
      while (status.status === "RUNNING") {
        await new Promise(r => setTimeout(r, 5000));
        const res = await fetch(`${HEX_BASE}/api/v1/run/${run.runId}/status`, { headers });
        status = await res.json();
      }
      results.push(status);
    }
    if (i + batchSize < projects.length) await new Promise(r => setTimeout(r, 15_000));
  }
  return results;
}

Error Handling

Prerequisites

• An approved request budget, observed header baseline, and sandbox run workload using opaque project IDs.
• A bounded queue, idempotency key for mutations, and reviewed recovery path for exhausted work.

Instructions

1. Classify requests by project and operation, applying per-scope concurrency limits before dispatch.
2. Honor server retry guidance when available; otherwise use bounded exponential backoff with jitter and a maximum attempt count.
3. Never replay a run, publish, or schedule mutation without its idempotency key, and defer nonessential work before backlog threatens freshness.
4. Monitor aggregate limited/deferred counts and synthetic canary success, tuning a single scope at a time with rollback.
5. Route exhausted work to reviewed recovery rather than silently dropping or duplicating runs.

Output

Return a rate-limit receipt with scope, requested/limited/deferred counts, retry revision, idempotency state, queue health, canary result, and rollback reference. Exclude query text, SQL, output, and credentials.

Examples

scope=sandbox-projects; requested=100; limited=3; deferred=3; retry=v2; idempotent=pass; queue=healthy; rollback=limits-r7 proves bounded handling.

Resources

• Hex API Documentation

Next Steps

See hex-performance-tuning.