Langfuse Rate Limits

Langfuse Rate Limits

Contents

• Overview
• Prerequisites
• Instructions
• Output
• Error Handling
• Examples
• Resources

Overview

Handle Langfuse rate limits gracefully with optimized batching, exponential backoff, concurrent request limiting, and sampling strategies for high-volume workloads.

Prerequisites

• Langfuse SDK installed
• Understanding of async/await patterns
• High-volume trace workload

Instructions

Step 1: Configure Optimal Batching

Set flushAt to 50 events and flushInterval to 5000ms. Use 30s request timeout for large batches.

Step 2: Implement Exponential Backoff

Create retry logic with configurable max retries (5), base delay (1s), max delay (30s), and jitter (500ms). Honor Retry-After headers. Only retry on 429 and 5xx errors.

Step 3: Build Rate Limit-Aware Wrapper

Wrap the Langfuse client with a concurrency limiter (max 100 pending events). Queue excess requests and release capacity on completion.

Step 4: Add Sampling for Ultra-High Volume

Implement configurable sampling (e.g., 10% of traces) with an alwaysSample predicate for errors and critical traces. Return no-op traces for unsampled requests.

Step 5: Monitor Rate Limit Usage

Track X-RateLimit-Remaining and X-RateLimit-Reset headers. Pre-emptively throttle when remaining requests drop below 10.

See detailed implementation for advanced patterns.

Output

• Optimized batching configuration
• Exponential backoff with jitter
• Concurrent request limiting
• Configurable sampling
• Rate limit monitoring

Error Handling

Examples

Rate Limit Tiers

Queue-Based Rate Limiting

import PQueue from "p-queue";
const queue = new PQueue({ concurrency: 10, interval: 1000, intervalCap: 50 });  # 1000: 1 second in ms
async function queuedTrace(params) {
  return queue.add(() => langfuse.trace(params));
}

Resources

• Langfuse Rate Limits
• p-queue Documentation
• Langfuse SDK Configuration