2809 Cost Tracking 6af248d5

Cost Tracking Integration

@cascadeflow/langchain offers two flexible cost tracking options to fit your workflow:

📊 Cost Tracking Providers

1. LangSmith (Default)

Use LangSmith's server-side cost calculation - the native LangChain ecosystem choice.

import { ChatOpenAI } from '@langchain/openai';
import { withCascade } from '@cascadeflow/langchain';

const cascade = withCascade({
  drafter: new ChatOpenAI({ modelName: 'gpt-5-nano' }),
  verifier: new ChatOpenAI({ modelName: 'gpt-5' }),
  // LangSmith is the default provider
  costTrackingProvider: 'langsmith', // Can omit this line
});

✅ Benefits:

• ✓ Automatic, always up-to-date pricing
• ✓ No pricing table maintenance needed
• ✓ Multi-modal cost tracking (text, images, caching, reasoning tokens)
• ✓ Integrated with LangSmith UI for visualization
• ✓ Native LangChain ecosystem integration

❌ Requirements:

• Requires LANGSMITH_API_KEY environment variable
• Requires network connectivity (costs calculated server-side)

📈 Viewing Costs:

Costs are visible in your LangSmith Dashboard. Token counts are automatically sent to LangSmith for server-side cost calculation.

const result = await cascade.invoke("Your query");
const stats = cascade.getLastCascadeResult();

console.log('Model Used:', stats.modelUsed);       // drafter or verifier
console.log('Quality:', stats.drafterQuality);     // 0-1 score
console.log('Latency:', stats.latencyMs);          // milliseconds

// ⚠️ Costs are $0 locally (calculated by LangSmith)
console.log('Local Cost:', stats.totalCost);       // 0 (see LangSmith UI)

2. CascadeFlow (Local Calculation)

Use CascadeFlow's built-in pricing table for offline, dependency-free cost tracking.

const cascade = withCascade({
  drafter: new ChatOpenAI({ modelName: 'gpt-5-nano' }),
  verifier: new ChatOpenAI({ modelName: 'gpt-5' }),
  costTrackingProvider: 'cascadeflow', // Use local pricing
});

✅ Benefits:

• ✓ No external dependencies
• ✓ Works offline
• ✓ Immediate local cost feedback
• ✓ No LangSmith account required
• ✓ Privacy-friendly (no data sent externally)

❌ Limitations:

• Pricing table may lag behind provider updates
• Limited to text tokens (no multi-modal support yet)
• Requires manual updates for new models

📈 Viewing Costs:

Costs are calculated immediately and returned in the stats object:

const result = await cascade.invoke("Your query");
const stats = cascade.getLastCascadeResult();

console.log('Drafter Cost:', stats.drafterCost);      // $0.000123
console.log('Verifier Cost:', stats.verifierCost);    // $0 (if not used)
console.log('Total Cost:', stats.totalCost);          // $0.000123
console.log('Savings:', stats.savingsPercentage);     // 66.7%

🎯 When to Use Each

Use LangSmith (default) when:

• ✅ You already use LangSmith for observability
• ✅ You want the most accurate, up-to-date pricing
• ✅ You need multi-modal cost tracking
• ✅ You want cost visualization in LangSmith UI
• ✅ You're deploying to production with LangChain ecosystem

Use CascadeFlow when:

• ✅ You don't want external dependencies
• ✅ You need offline support
• ✅ You want immediate local cost feedback
• ✅ You're prototyping and don't have LangSmith yet
• ✅ You prefer privacy-focused, local-only tracking

⚙️ Configuration

interface CascadeConfig {
  drafter: BaseChatModel;
  verifier: BaseChatModel;
  qualityThreshold?: number;               // 0-1, default: 0.7
  enableCostTracking?: boolean;            // default: true
  costTrackingProvider?: 'langsmith' | 'cascadeflow';  // default: 'langsmith'
}

Disabling Cost Tracking

const cascade = withCascade({
  drafter,
  verifier,
  enableCostTracking: false, // Disable all cost tracking
});

🔄 Switching Providers

You can easily switch between providers:

// Development: Use CascadeFlow for quick local feedback
const devCascade = withCascade({
  drafter,
  verifier,
  costTrackingProvider: 'cascadeflow',
});

// Production: Use LangSmith for comprehensive observability
const prodCascade = withCascade({
  drafter,
  verifier,
  costTrackingProvider: 'langsmith',
});

📦 Supported Models (CascadeFlow Provider)

When using costTrackingProvider: 'cascadeflow', the following models have built-in pricing:

OpenAI

• gpt-5, gpt-5-mini, gpt-5-nano, gpt-5.1
• gpt-4o, gpt-4o-mini, gpt-4-turbo
• gpt-3.5-turbo

Anthropic

• claude-3-5-sonnet-20241022, claude-3-5-haiku-20241022
• claude-3-opus-20240229, claude-3-sonnet-20240229, claude-3-haiku-20240307
• claude-sonnet-4, claude-haiku-4.5

Google

• gemini-2.5-flash, gemini-2.5-pro
• gemini-1.5-pro, gemini-1.5-flash

Others

• Groq models
• Together AI models

Missing a model? It will default to $0 cost. Open an issue to request additions.

🌐 Environment Variables

For LangSmith Provider

# Required for LangSmith cost tracking
LANGSMITH_API_KEY=lsv2_pt_...
LANGSMITH_PROJECT=your-project-name
LANGSMITH_TRACING=true  # Optional, enables full tracing

For CascadeFlow Provider

No environment variables required! Works completely offline.

💡 Best Practices

1. Development: Use cascadeflow provider for fast iteration without LangSmith dependency
2. Production: Use langsmith provider for comprehensive cost tracking and observability
3. Hybrid: Use cascadeflow locally, langsmith in CI/production
4. Privacy: Use cascadeflow if you can't send data to external services

📊 Example: Full Comparison

See examples/cost-tracking-providers.ts for a complete side-by-side demonstration of both providers.

npx tsx examples/cost-tracking-providers.ts

🔗 Related Documentation

• LangSmith Documentation
• CascadeFlow Pricing Table
• Main README