Cohere CI Integration

Cohere CI Integration

Overview

Set up CI/CD pipelines with automated unit tests (mocked) and integration tests (real API) for Cohere API v2 applications.

Prerequisites

• GitHub repository with Actions enabled
• Cohere trial or production API key
• cohere-ai package installed

Instructions

Step 1: GitHub Actions Workflow

# .github/workflows/cohere-ci.yml
name: Cohere CI

on:
  push:
    branches: [main]
  pull_request:
    branches: [main]

jobs:
  unit-tests:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: '20'
          cache: 'npm'
      - run: npm ci
      - run: npm test -- --coverage
        # Unit tests use mocked Cohere responses — no API key needed

  integration-tests:
    runs-on: ubuntu-latest
    if: github.event_name == 'push' && github.ref == 'refs/heads/main'
    env:
      CO_API_KEY: ${{ secrets.CO_API_KEY }}
    steps:
      - uses: actions/checkout@v4
      - uses: actions/setup-node@v4
        with:
          node-version: '20'
          cache: 'npm'
      - run: npm ci
      - name: Run Cohere integration tests
        run: npm run test:integration
        timeout-minutes: 5

Step 2: Configure GitHub Secrets

# Store Cohere API key as a repo secret
gh secret set CO_API_KEY --body "your-production-key-here"

# Verify it was set
gh secret list

Step 3: Unit Tests (Mocked — No API Key)

// tests/unit/cohere-chat.test.ts
import { describe, it, expect, vi, beforeEach } from 'vitest';

// Mock the entire SDK
vi.mock('cohere-ai', () => ({
  CohereClientV2: vi.fn().mockImplementation(() => ({
    chat: vi.fn().mockResolvedValue({
      message: { content: [{ type: 'text', text: 'mocked response' }] },
      finishReason: 'COMPLETE',
      usage: { billedUnits: { inputTokens: 5, outputTokens: 3 } },
    }),
    embed: vi.fn().mockResolvedValue({
      embeddings: { float: [[0.1, 0.2, 0.3]] },
    }),
    rerank: vi.fn().mockResolvedValue({
      results: [{ index: 0, relevanceScore: 0.95 }],
    }),
  })),
}));

describe('Chat service', () => {
  it('returns text from chat completion', async () => {
    const { CohereClientV2 } = await import('cohere-ai');
    const cohere = new CohereClientV2();

    const response = await cohere.chat({
      model: 'command-a-03-2025',
      messages: [{ role: 'user', content: 'test' }],
    });

    expect(response.message?.content?.[0]?.text).toBe('mocked response');
    expect(cohere.chat).toHaveBeenCalledWith(
      expect.objectContaining({ model: 'command-a-03-2025' })
    );
  });

  it('handles embed with correct input type', async () => {
    const { CohereClientV2 } = await import('cohere-ai');
    const cohere = new CohereClientV2();

    const response = await cohere.embed({
      model: 'embed-v4.0',
      texts: ['test'],
      inputType: 'search_document',
      embeddingTypes: ['float'],
    });

    expect(response.embeddings.float).toHaveLength(1);
  });
});

Step 4: Integration Tests (Real API — Gated)

// tests/integration/cohere.test.ts
import { describe, it, expect } from 'vitest';
import { CohereClientV2 } from 'cohere-ai';

const hasApiKey = !!process.env.CO_API_KEY;

describe.skipIf(!hasApiKey)('Cohere Integration', () => {
  const cohere = new CohereClientV2();

  it('chat completion works', async () => {
    const response = await cohere.chat({
      model: 'command-r7b-12-2024', // cheapest model for CI
      messages: [{ role: 'user', content: 'Reply with exactly: OK' }],
      maxTokens: 5,
    });
    expect(response.message?.content?.[0]?.text).toBeTruthy();
    expect(response.finishReason).toBe('COMPLETE');
  }, 15_000);

  it('embed generates vectors', async () => {
    const response = await cohere.embed({
      model: 'embed-v4.0',
      texts: ['CI test embedding'],
      inputType: 'search_document',
      embeddingTypes: ['float'],
    });
    expect(response.embeddings.float[0].length).toBeGreaterThan(0);
  }, 15_000);

  it('rerank scores documents', async () => {
    const response = await cohere.rerank({
      model: 'rerank-v3.5',
      query: 'machine learning',
      documents: ['ML is AI', 'cooking recipes', 'deep learning'],
      topN: 2,
    });
    expect(response.results).toHaveLength(2);
    expect(response.results[0].relevanceScore).toBeGreaterThan(0.5);
  }, 15_000);
});

Step 5: Package Scripts

{
  "scripts": {
    "test": "vitest --run",
    "test:watch": "vitest --watch",
    "test:integration": "COHERE_INTEGRATION=1 vitest --run tests/integration/",
    "test:coverage": "vitest --run --coverage"
  }
}

CI Cost Control

• Use command-r7b-12-2024 (cheapest) for integration tests
• Set maxTokens: 5 on all CI chat calls
• Run integration tests only on main pushes (not PRs)
• Use trial key for CI if under 1000 monthly calls

Error Handling

Output

• A pull-request lane that runs deterministic mocked tests without a Cohere credential.
• A protected-main integration lane that uses the repository secret without printing it, and reports pass/fail, coverage, and the failed test name in GitHub Actions.
• A cost-bounded live check: the job has a five-minute ceiling, uses minimal prompts, and is not triggered by untrusted pull-request code.

Examples

For a release candidate, run the low-cost integration suite from a trusted local environment after setting CO_API_KEY in the shell (never in a command line, log, or committed file):

npm test
npm run test:integration

If the integration job returns a 429, keep the unit-test result as the merge signal, record the rate-limit response in the incident log with secrets redacted, and retry the protected-main job after the documented backoff window. Do not add the key to a pull-request workflow or weaken the branch protection to work around a transient API failure.

Resources

• GitHub Actions Secrets
• Vitest Configuration
• Cohere Rate Limits

Next Steps

For deployment patterns, see cohere-deploy-integration.