# Token Cost Computation

> Compute token usage totals and costs from Codex CLI and Claude Code SSE JSONL logs. Use when user asks about token consumption, API costs, usage breakdown, billing estimates, or analyzing SSE logs for token metrics.

- Skill: `tankygranny05/token-cost-computation` (Agent Skill, multi-file: 4 files)
- Install (CLI): `npx skillmds@latest add tankygranny05/token-cost-computation`
- Raw SKILL.md: https://api.skillmd.com/api/skills/tankygranny05/token-cost-computation/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: tankygranny05 (https://skillmd.com/u/tankygranny05)
- Updated: 2026-09-10
- Page: https://skillmd.com/skills/tankygranny05/token-cost-computation

---

<!-- [Created by Claude: 5e523850-645e-45e1-97bf-797fe2aa8669] -->

# Token Cost Computation

Compute token usage totals and estimated costs from **Codex CLI** (OpenAI) and **Claude Code** (Anthropic) SSE JSONL log streams.

## Quick Start

### Compute Token Costs from SSE Logs

```bash
# Basic usage with both log types
python scripts/compute_token_costs.py \
  --codex ~/centralized-logs/codex/sse_lines.jsonl \
  --claude ~/centralized-logs/claude/sse_lines.jsonl

# With a pricing snapshot for reproducibility
python scripts/compute_token_costs.py \
  --codex codex_sse_lines.jsonl \
  --claude claude_sse_lines.jsonl \
  --pricing-snapshot pricing_snapshots/pricing_snapshot_2026-01-24.json
```

### Fetch Latest Pricing

```bash
# Fetch and save a dated pricing snapshot
python scripts/pull_pricing.py --snapshot-dir pricing_snapshots/ --print-summary
```

### Extract a Time Window from Logs

```bash
# Extract a specific week from Codex logs
python scripts/extract_codex_week.py \
  --src ~/centralized-logs/codex/sse_lines.jsonl \
  --dst extracted_week.jsonl \
  --start "2026-01-15 14:47" \
  --end "2026-01-22 14:47"
```

## What It Computes

### Token Metrics

| Provider | Metric | Description |
|----------|--------|-------------|
| Codex (OpenAI) | `input_tokens` | Total input tokens across all calls |
| Codex (OpenAI) | `cached_input_tokens` | Subset of input tokens served from cache |
| Codex (OpenAI) | `output_tokens` | Total output tokens generated |
| Codex (OpenAI) | `reasoning_output_tokens` | Tokens used for reasoning (o-series models) |
| Claude (Anthropic) | `input_tokens` | Total input tokens |
| Claude (Anthropic) | `cache_creation_input_tokens` | Tokens written to cache |
| Claude (Anthropic) | `cache_read_input_tokens` | Tokens read from cache |
| Claude (Anthropic) | `output_tokens` | Total output tokens |

### Cost Estimation

Costs are computed using:
- **OpenAI**: Standard/Flex/Priority tier pricing per model
- **Anthropic**: Base pricing + cache multipliers (1.25x write-5m, 2.0x write-1h, 0.1x read)

## Key Concepts

### Token Aggregation Semantics

**Codex (OpenAI):**
- Uses `last_token_usage` from `turn.token_count` events (not cumulative totals)
- Deduplicates consecutive identical events to avoid double-counting
- `cached_input_tokens` is a **subset** of `input_tokens`, not additive

**Claude (Anthropic):**
- Uses **max** of `message_start` and `message_delta` usage per message (they're snapshots, not deltas)
- Cache pricing is writes (1.25x or 2x) + reads (0.1x), NOT a flat 90% discount

### Pricing Sources (in order of reliability)

1. **OpenAI**: `https://platform.openai.com/docs/pricing`
2. **Anthropic**: `https://platform.claude.com/docs/en/about-claude/pricing`
3. **OpenRouter**: `https://openrouter.ai/api/v1/models` (machine-readable, requires API key)

## Output Format

### Console Output

```
=== OpenAI / Codex ===
Model: gpt-5.2 | tier=standard | pricing source: snapshot:...
API calls counted (deduped token_count): 150
input_tokens=8,947,835 (cached=8,656,512, noncached=291,323)
output_tokens=65,051 (reasoning_output_tokens=0)
estimated cost: $1.42

=== Anthropic / Claude ===
Model: claude-opus-4-5 | pricing source: snapshot:...
  messages=75 | input=2,500,000 | cache_write=500,000 | cache_read=2,000,000 | output=150,000
  estimated cost: $4.25

=== Totals ===
Grand total: $5.67
```

### JSON Output

The script also outputs a machine-readable JSON structure with full breakdown.

## Scripts Reference

| Script | Purpose |
|--------|---------|
| `compute_token_costs.py` | Main computation - totals + costs |
| `pull_pricing.py` | Fetch latest pricing from providers |
| `extract_codex_week.py` | Extract time windows from SSE logs |

## Important Notes

- **Do NOT** hardcode "90% cached discount" - varies by provider/model/tier
- **Do NOT** sum `message_start` + `message_delta` usage - they're snapshots, not increments
- **Do NOT** use `total_token_usage` for per-round deltas without accounting for session resume
- Use `--pricing-snapshot` for reproducible cost audits

## Context Window Consumption

**Note:** This skill computes token **totals** and **costs**, not context window utilization. The raw logs contain `model_context_window` data, but utilization percentage is not currently computed.

To compute context utilization, you would need:
```python
utilization_pct = (total_tokens_at_turn / model_context_window) * 100
```

## Related Locations

- Centralized Codex logs: `~/centralized-logs/codex/sse_lines.jsonl`
- Centralized Claude logs: `~/centralized-logs/claude/sse_lines.jsonl`
- Source repo: `~/swe/token-computation/`