Token Cost Computation

Token Cost Computation

Compute token usage totals and estimated costs from Codex CLI (OpenAI) and Claude Code (Anthropic) SSE JSONL log streams.

Quick Start

Compute Token Costs from SSE Logs

# Basic usage with both log types
python scripts/compute_token_costs.py \
  --codex ~/centralized-logs/codex/sse_lines.jsonl \
  --claude ~/centralized-logs/claude/sse_lines.jsonl

# With a pricing snapshot for reproducibility
python scripts/compute_token_costs.py \
  --codex codex_sse_lines.jsonl \
  --claude claude_sse_lines.jsonl \
  --pricing-snapshot pricing_snapshots/pricing_snapshot_2026-01-24.json

Fetch Latest Pricing

# Fetch and save a dated pricing snapshot
python scripts/pull_pricing.py --snapshot-dir pricing_snapshots/ --print-summary

Extract a Time Window from Logs

# Extract a specific week from Codex logs
python scripts/extract_codex_week.py \
  --src ~/centralized-logs/codex/sse_lines.jsonl \
  --dst extracted_week.jsonl \
  --start "2026-01-15 14:47" \
  --end "2026-01-22 14:47"

What It Computes

Token Metrics

Cost Estimation

Costs are computed using:

• OpenAI: Standard/Flex/Priority tier pricing per model
• Anthropic: Base pricing + cache multipliers (1.25x write-5m, 2.0x write-1h, 0.1x read)

Key Concepts

Token Aggregation Semantics

Codex (OpenAI):

• Uses last_token_usage from turn.token_count events (not cumulative totals)
• Deduplicates consecutive identical events to avoid double-counting
• cached_input_tokens is a subset of input_tokens, not additive

Claude (Anthropic):

• Uses max of message_start and message_delta usage per message (they're snapshots, not deltas)
• Cache pricing is writes (1.25x or 2x) + reads (0.1x), NOT a flat 90% discount

Pricing Sources (in order of reliability)

1. OpenAI: https://platform.openai.com/docs/pricing
2. Anthropic: https://platform.claude.com/docs/en/about-claude/pricing
3. OpenRouter: https://openrouter.ai/api/v1/models (machine-readable, requires API key)

Output Format

Console Output

=== OpenAI / Codex ===
Model: gpt-5.2 | tier=standard | pricing source: snapshot:...
API calls counted (deduped token_count): 150
input_tokens=8,947,835 (cached=8,656,512, noncached=291,323)
output_tokens=65,051 (reasoning_output_tokens=0)
estimated cost: $1.42

=== Anthropic / Claude ===
Model: claude-opus-4-5 | pricing source: snapshot:...
  messages=75 | input=2,500,000 | cache_write=500,000 | cache_read=2,000,000 | output=150,000
  estimated cost: $4.25

=== Totals ===
Grand total: $5.67

JSON Output

The script also outputs a machine-readable JSON structure with full breakdown.

Scripts Reference

Important Notes

• Do NOT hardcode "90% cached discount" - varies by provider/model/tier
• Do NOT sum message_start + message_delta usage - they're snapshots, not increments
• Do NOT use total_token_usage for per-round deltas without accounting for session resume
• Use --pricing-snapshot for reproducible cost audits

Context Window Consumption

Note: This skill computes token totals and costs, not context window utilization. The raw logs contain model_context_window data, but utilization percentage is not currently computed.

To compute context utilization, you would need:

utilization_pct = (total_tokens_at_turn / model_context_window) * 100

Related Locations

• Centralized Codex logs: ~/centralized-logs/codex/sse_lines.jsonl
• Centralized Claude logs: ~/centralized-logs/claude/sse_lines.jsonl
• Source repo: ~/swe/token-computation/