Cohere Prod Checklist

Cohere Production Checklist

Overview

Complete go-live checklist for deploying Cohere API v2 integrations to production with safety gates, health checks, and rollback procedures.

Prerequisites

• Staging environment tested and verified
• Production API key (not trial) from dashboard.cohere.com
• Deployment pipeline configured
• Monitoring and alerting ready

Checklist

API & Authentication

• Using production API key (not trial — trial is rate-limited to 20 calls/min)
• CO_API_KEY stored in secret manager (Vault, AWS Secrets Manager, GCP Secret Manager)
• Key rotation procedure documented and tested
• Billing alerts configured at dashboard.cohere.com
• Using API v2 endpoints (CohereClientV2, not CohereClient)

Code Quality

• All API calls specify model parameter explicitly
• embeddingTypes set for all Embed calls (required for v3+)
• inputType set for all Embed calls (required for v3+)
• Error handling catches CohereError and CohereTimeoutError
• Retry logic with exponential backoff for 429 and 5xx
• No hardcoded API keys in source code
• Request/response logging excludes API keys and PII

Model Selection

• Correct model IDs used (not deprecated names):

Performance

• Embed calls batched (up to 96 texts per request)
• Rerank calls limited to 1000 documents per request
• Streaming enabled for user-facing chat (chatStream)
• Connection pooling / keep-alive configured
• Response caching for repeated embed/rerank queries
• maxTokens set to prevent runaway generation costs

Health Check Endpoint

// /api/health
import { CohereClientV2, CohereError } from 'cohere-ai';

const cohere = new CohereClientV2();

export async function GET() {
  const start = Date.now();
  let cohereStatus: 'healthy' | 'degraded' | 'down' = 'down';

  try {
    // Cheapest possible health check — minimal chat
    await cohere.chat({
      model: 'command-r7b-12-2024',
      messages: [{ role: 'user', content: 'ping' }],
      maxTokens: 1,
    });
    cohereStatus = 'healthy';
  } catch (err) {
    if (err instanceof CohereError && err.statusCode === 429) {
      cohereStatus = 'degraded'; // Rate limited but reachable
    }
  }

  return Response.json({
    status: cohereStatus === 'healthy' ? 'ok' : 'degraded',
    cohere: {
      status: cohereStatus,
      latencyMs: Date.now() - start,
    },
    timestamp: new Date().toISOString(),
  });
}

Circuit Breaker

class CohereCircuitBreaker {
  private failures = 0;
  private lastFailure = 0;
  private state: 'closed' | 'open' | 'half-open' = 'closed';

  constructor(
    private threshold = 5,
    private resetMs = 60_000
  ) {}

  async call<T>(fn: () => Promise<T>, fallback?: () => T): Promise<T> {
    if (this.state === 'open') {
      if (Date.now() - this.lastFailure > this.resetMs) {
        this.state = 'half-open';
      } else if (fallback) {
        return fallback();
      } else {
        throw new Error('Cohere circuit breaker is open');
      }
    }

    try {
      const result = await fn();
      this.failures = 0;
      this.state = 'closed';
      return result;
    } catch (err) {
      this.failures++;
      this.lastFailure = Date.now();

      if (this.failures >= this.threshold) {
        this.state = 'open';
        console.error(`Cohere circuit breaker OPEN after ${this.failures} failures`);
      }
      throw err;
    }
  }
}

const breaker = new CohereCircuitBreaker();

Gradual Rollout

# Pre-flight
curl -sf https://staging.example.com/api/health | jq '.cohere'
curl -s https://status.cohere.com/api/v2/status.json | jq '.status'

# Deploy with canary (10% traffic)
kubectl apply -f k8s/production.yaml
kubectl rollout pause deployment/app

# Monitor for 10 minutes: error rate, latency, 429s
# Check: No increase in CohereError rate
# Check: P95 latency < 5s for chat, < 500ms for embed/rerank

# Proceed to 100%
kubectl rollout resume deployment/app
kubectl rollout status deployment/app

Monitoring Alerts

Rollback

# Immediate rollback
kubectl rollout undo deployment/app
kubectl rollout status deployment/app

# Verify rollback
curl -sf https://api.example.com/api/health | jq '.cohere'

Instructions

Complete the production checklist in sequence: verify scoped secret and model configuration, run a bounded health request, test circuit-breaker, budget, and alert behavior, then obtain the documented approval before enabling user traffic. Capture evidence for rollback and data/safety controls; an available endpoint alone is not production acceptance.

Error Handling

Examples

Before release, send a synthetic staging request through the deployed path, verify health, alerts, circuit breaker, and rollback, and attach redacted evidence to the change record. If any control fails, retain the prior certified release and correct it before promotion.

Output

• Production-ready Cohere integration with health checks
• Circuit breaker preventing cascade failures
• Monitoring alerts for Cohere-specific error conditions
• Documented rollback procedure

Resources

• Cohere Going Live Guide
• Cohere Status Page
• Cohere Pricing

Next Steps

For version upgrades, see cohere-upgrade-migration.