Exa Observability

Exa Observability

Overview

Monitor Exa AI search API performance, result quality, and cost efficiency. Key metrics include search latency (Exa neural search typically takes 500-2000ms), result relevance (measured by click-through or downstream usage), search volume by type (neural vs keyword vs auto), per-search cost tracking, and cache hit rates for repeated queries.

Prerequisites

• Exa API integration in production
• Metrics backend (Prometheus, Datadog, or equivalent)
• Request logging infrastructure

Instructions

Step 1: Instrument the Exa Client

import Exa from 'exa-js';

async function trackedSearch(exa: Exa, query: string, options: any) {
  const start = performance.now();
  try {
    const results = await exa.search(query, options);
    const duration = performance.now() - start;
    emitHistogram('exa_search_duration_ms', duration, { type: options.type || 'auto' });
    emitCounter('exa_searches_total', 1, { type: options.type || 'auto', status: 'success' });
    emitGauge('exa_results_count', results.results.length, { type: options.type || 'auto' });
    return results;
  } catch (err: any) {
    emitCounter('exa_searches_total', 1, { status: 'error', code: err.status });
    throw err;
  }
}

Step 2: Track Result Quality

// Measure whether search results are actually used by downstream consumers
function trackResultUsage(searchId: string, resultIndex: number, action: 'clicked' | 'used_in_context' | 'discarded') {
  emitCounter('exa_result_usage', 1, { action, position: String(resultIndex) });
  // Results at position 0-2 should have high usage; if not, query needs tuning
}

Step 3: Monitor Search Budget

set -euo pipefail
# Check remaining search quota
curl -s https://api.exa.ai/v1/usage \
  -H "x-api-key: $EXA_API_KEY" | \
  jq '{searches_today, searches_this_month, monthly_limit, budget_remaining_pct: (1 - .searches_this_month / .monthly_limit) * 100}'

Step 4: Configure Alerts

groups:
  - name: exa
    rules:
      - alert: ExaHighLatency
        expr: histogram_quantile(0.95, rate(exa_search_duration_ms_bucket[5m])) > 3000  # 3000: 3 seconds in ms
        annotations: { summary: "Exa search P95 latency exceeds 3 seconds" }
      - alert: ExaBudgetLow
        expr: exa_monthly_searches_remaining < 1000  # 1000: 1 second in ms
        annotations: { summary: "Exa monthly search budget nearly exhausted" }
      - alert: ExaLowResultQuality
        expr: rate(exa_result_usage{action="discarded"}[1h]) / rate(exa_result_usage[1h]) > 0.5
        annotations: { summary: "Over 50% of Exa search results being discarded" }
      - alert: ExaApiErrors
        expr: rate(exa_searches_total{status="error"}[5m]) > 0.1
        annotations: { summary: "Exa API errors detected" }

Step 5: Build a Search Efficiency Dashboard

Key panels: search volume by type (neural/keyword/auto), latency p50/p95, results per search distribution, result usage rate (used vs discarded), daily cost tracking, and cache hit rate. Low result counts with high latency indicate poorly formed queries.

Error Handling

Examples

Basic usage: Apply exa observability to a standard project setup with default configuration options.

Advanced scenario: Customize exa observability for production environments with multiple constraints and team-specific requirements.

Output

• Configuration files or code changes applied to the project
• Validation report confirming correct implementation
• Summary of changes made and their rationale

Resources

• Official monitoring documentation
• Community best practices and patterns
• Related skills in this plugin pack