Firecrawl Multi Env Setup

FireCrawl Multi-Environment Setup

Overview

Configure FireCrawl across development, staging, and production environments.

Prerequisites

• Separate FireCrawl accounts or API keys per environment
• Secret management solution (Vault, AWS Secrets Manager, etc.)
• CI/CD pipeline with environment variables
• Environment detection in application

Environment Strategy

Configuration Structure

config/
├── firecrawl/
│   ├── base.json           # Shared config
│   ├── development.json    # Dev overrides
│   ├── staging.json        # Staging overrides
│   └── production.json     # Prod overrides

base.json

{
  "timeout": 30000,
  "retries": 3,
  "cache": {
    "enabled": true,
    "ttlSeconds": 60
  }
}

development.json

{
  "apiKey": "${FIRECRAWL_API_KEY}",
  "baseUrl": "https://api-sandbox.firecrawl.com",
  "debug": true,
  "cache": {
    "enabled": false
  }
}

staging.json

{
  "apiKey": "${FIRECRAWL_API_KEY_STAGING}",
  "baseUrl": "https://api-staging.firecrawl.com",
  "debug": false
}

production.json

{
  "apiKey": "${FIRECRAWL_API_KEY_PROD}",
  "baseUrl": "https://api.firecrawl.com",
  "debug": false,
  "retries": 5
}

Environment Detection

// src/firecrawl/config.ts
import baseConfig from '../../config/firecrawl/base.json';

type Environment = 'development' | 'staging' | 'production';

function detectEnvironment(): Environment {
  const env = process.env.NODE_ENV || 'development';
  const validEnvs: Environment[] = ['development', 'staging', 'production'];
  return validEnvs.includes(env as Environment)
    ? (env as Environment)
    : 'development';
}

export function getFireCrawlConfig() {
  const env = detectEnvironment();
  const envConfig = require(`../../config/firecrawl/${env}.json`);

  return {
    ...baseConfig,
    ...envConfig,
    environment: env,
  };
}

Secret Management by Environment

Local Development

# .env.local (git-ignored)
FIRECRAWL_API_KEY=sk_test_dev_***

CI/CD (GitHub Actions)

env:
  FIRECRAWL_API_KEY: ${{ secrets.FIRECRAWL_API_KEY_${{ matrix.environment }} }}

Production (Vault/Secrets Manager)

# AWS Secrets Manager
aws secretsmanager get-secret-value --secret-id firecrawl/production/api-key

# GCP Secret Manager
gcloud secrets versions access latest --secret=firecrawl-api-key

# HashiCorp Vault
vault kv get -field=api_key secret/firecrawl/production

Environment Isolation

// Prevent production operations in non-prod
function guardProductionOperation(operation: string): void {
  const config = getFireCrawlConfig();

  if (config.environment !== 'production') {
    console.warn(`[firecrawl] ${operation} blocked in ${config.environment}`);
    throw new Error(`${operation} only allowed in production`);
  }
}

// Usage
async function deleteAllData() {
  guardProductionOperation('deleteAllData');
  // Dangerous operation here
}

Feature Flags by Environment

const featureFlags: Record<Environment, Record<string, boolean>> = {
  development: {
    newFeature: true,
    betaApi: true,
  },
  staging: {
    newFeature: true,
    betaApi: false,
  },
  production: {
    newFeature: false,
    betaApi: false,
  },
};

Instructions

Step 1: Create Config Structure

Set up the base and per-environment configuration files.

Step 2: Implement Environment Detection

Add logic to detect and load environment-specific config.

Step 3: Configure Secrets

Store API keys securely using your secret management solution.

Step 4: Add Environment Guards

Implement safeguards for production-only operations.

Output

• Multi-environment config structure
• Environment detection logic
• Secure secret management
• Production safeguards enabled

Error Handling

Examples

Quick Environment Check

const env = getFireCrawlConfig();
console.log(`Running in ${env.environment} with ${env.baseUrl}`);

Resources

• FireCrawl Environments Guide
• 12-Factor App Config

Next Steps

For observability setup, see firecrawl-observability.