Openevidence Rate Limits

OpenEvidence Rate Limits

Overview

Rate limit management for OpenEvidence clinical query API. Clinical decision support queries are latency-sensitive but also expensive (web-scale medical literature search per query), requiring careful throughput management.

Prerequisites

• OpenEvidence API access
• Understanding of clinical workflow request patterns
• Redis for rate tracking (recommended)

OpenEvidence Rate Limits

Instructions

Step 1: Per-Endpoint Rate Limiter

Clinical queries and DeepConsult have different limits. Track separately.

class OpenEvidenceRateLimiter {
  private limits: Record<string, { rpm: number; concurrent: number }> = {
    'query': { rpm: 60, concurrent: 10 },
    'deepconsult': { rpm: 10, concurrent: 2 },
    'feedback': { rpm: 120, concurrent: 20 },
  };
  private active: Map<string, number> = new Map();
  private requests: Map<string, number[]> = new Map();

  async acquire(endpoint: string): Promise<void> {
    const limit = this.limits[endpoint];
    if (!limit) throw new Error(`Unknown endpoint: ${endpoint}`);

    // Check concurrent
    const active = this.active.get(endpoint) || 0;
    if (active >= limit.concurrent) {
      await this.waitForSlot(endpoint);
    }

    // Check RPM
    const now = Date.now();
    const times = (this.requests.get(endpoint) || []).filter(t => now - t < 60000);  # 60000: 1 minute in ms
    if (times.length >= limit.rpm) {
      const waitMs = 60000 - (now - times[0]);  # 1 minute in ms
      await new Promise(r => setTimeout(r, waitMs + 100));
    }

    this.active.set(endpoint, (this.active.get(endpoint) || 0) + 1);
    times.push(Date.now());
    this.requests.set(endpoint, times);
  }

  release(endpoint: string): void {
    this.active.set(endpoint, Math.max(0, (this.active.get(endpoint) || 0) - 1));
  }
}

Step 2: Clinical Query Priority Queue

Stat queries should preempt routine queries.

class ClinicalQueryQueue {
  private statQueue: Array<() => Promise<any>> = [];
  private routineQueue: Array<() => Promise<any>> = [];
  private limiter = new OpenEvidenceRateLimiter();

  async submit(query: any, urgency: 'stat' | 'routine'): Promise<any> {
    return new Promise((resolve, reject) => {
      const task = async () => {
        await this.limiter.acquire('query');
        try {
          const result = await openevidence.query(query);
          resolve(result);
        } catch (e) { reject(e); }
        finally { this.limiter.release('query'); }
      };
      if (urgency === 'stat') this.statQueue.push(task);
      else this.routineQueue.push(task);
      this.process();
    });
  }

  private async process() {
    const task = this.statQueue.shift() || this.routineQueue.shift();
    if (task) await task();
  }
}

Step 3: Backoff Strategy for 429 Responses

async function queryWithBackoff(query: any, maxRetries = 3): Promise<any> {
  for (let attempt = 0; attempt <= maxRetries; attempt++) {
    try {
      return await openevidence.query(query);
    } catch (e: any) {
      if (e.status === 429 && attempt < maxRetries) {  # HTTP 429 Too Many Requests
        const retryAfter = parseInt(e.headers?.['retry-after'] || '5');
        await new Promise(r => setTimeout(r, retryAfter * 1000));  # 1000: 1 second in ms
      } else { throw e; }
    }
  }
}

Error Handling

Examples

Basic usage: Apply openevidence rate limits to a standard project setup with default configuration options.

Advanced scenario: Customize openevidence rate limits for production environments with multiple constraints and team-specific requirements.

Resources

• OpenEvidence API

Output

• Configuration files or code changes applied to the project
• Validation report confirming correct implementation
• Summary of changes made and their rationale