Glean Rate Limits

Glean Rate Limits

Overview

Glean's APIs split into two tiers: the Indexing API for pushing documents into the search corpus, and the Client API for executing searches. The Indexing API handles bulk document ingestion at approximately 100 requests per minute, while search queries are capped at 60 per minute per token. Organizations indexing large knowledge bases (100K+ documents from Confluence, Notion, or internal wikis) must implement careful batching to avoid 429 responses that can stall multi-hour ingestion pipelines.

Rate Limit Reference

Rate Limiter Implementation

class GleanRateLimiter {
  private tokens: number;
  private lastRefill: number;
  private readonly max: number;
  private readonly refillRate: number;
  private queue: Array<{ resolve: () => void }> = [];

  constructor(maxPerMinute: number) {
    this.max = maxPerMinute;
    this.tokens = maxPerMinute;
    this.lastRefill = Date.now();
    this.refillRate = maxPerMinute / 60_000;
  }

  async acquire(): Promise<void> {
    this.refill();
    if (this.tokens >= 1) { this.tokens -= 1; return; }
    return new Promise(resolve => this.queue.push({ resolve }));
  }

  private refill() {
    const now = Date.now();
    this.tokens = Math.min(this.max, this.tokens + (now - this.lastRefill) * this.refillRate);
    this.lastRefill = now;
    while (this.tokens >= 1 && this.queue.length) {
      this.tokens -= 1;
      this.queue.shift()!.resolve();
    }
  }
}

const indexLimiter = new GleanRateLimiter(18);  // buffer under 20 bulk/min
const searchLimiter = new GleanRateLimiter(50);

Retry Strategy

async function gleanRetry<T>(
  limiter: GleanRateLimiter, fn: () => Promise<Response>, maxRetries = 4
): Promise<T> {
  for (let attempt = 0; attempt <= maxRetries; attempt++) {
    await limiter.acquire();
    const res = await fn();
    if (res.ok) return res.json();
    if (res.status === 429) {
      const retryAfter = parseInt(res.headers.get("Retry-After") || "30", 10);
      const jitter = Math.random() * 5000;
      await new Promise(r => setTimeout(r, retryAfter * 1000 + jitter));
      continue;
    }
    if (res.status >= 500 && attempt < maxRetries) {
      await new Promise(r => setTimeout(r, Math.pow(2, attempt) * 2000));
      continue;
    }
    throw new Error(`Glean API ${res.status}: ${await res.text()}`);
  }
  throw new Error("Max retries exceeded");
}

Batch Processing

async function bulkIndexDocuments(docs: any[], batchSize = 100) {
  const results: any[] = [];
  for (let i = 0; i < docs.length; i += batchSize) {
    const batch = docs.slice(i, i + batchSize);
    const result = await gleanRetry(indexLimiter, () =>
      fetch(`${GLEAN_BASE}/api/index/v1/bulkindexdocuments`, {
        method: "POST", headers,
        body: JSON.stringify({ uploadId: `batch-${i}`, documents: batch }),
      })
    );
    results.push(result);
    if (i + batchSize < docs.length) await new Promise(r => setTimeout(r, 4000));
  }
  return results;
}

Error Handling

Prerequisites

• An approved per-operation budget, observed response-header baseline, and a test environment with synthetic requests.
• Idempotency keys for writes, a bounded queue, and a dead-letter path for exhausted retries.
• A defined caller identity and datasource scope so throttling cannot be bypassed by broadening credentials.

Instructions

1. Classify calls as read or write, apply per-scope concurrency limits, and honor server retry guidance when it is present.
2. Use bounded exponential backoff with jitter and a maximum attempt count; never blindly replay a write without its idempotency key.
3. Shed or defer nonessential work before queue growth threatens freshness or downstream access-control synchronization.
4. Monitor aggregate rate-limit responses and canary success, then tune one scope at a time with a rollback to the prior limits.
5. Route exhausted items to reviewed recovery rather than silently dropping or duplicating them.

Output

Return a rate-limit receipt with operation scope, request/limited/deferred counts, retry policy revision, idempotency outcome, queue state, canary result, and rollback reference. Omit tokens, URLs with sensitive parameters, and payloads.

Examples

scope=sandbox-index; requested=100; limited=3; deferred=3; retry=v2; idempotent=pass; queue=healthy; rollback=limits-r7 proves bounded handling.

Resources

• Glean Developer Portal
• Indexing API
• Search API

Next Steps

See glean-performance-tuning.