Analyze Model

analyze-model

Queries OpenRouter and Artificial Analysis APIs to build a live model catalog, then uses that catalog as the source of truth for cost analysis and usage forecasting based on real usage logs.

Authentication Setup

API keys resolved in this order for each provider:

1. .env file in the current workspace
2. Shell environment variable

Workflows

analyze-model fetch-catalog

Fetches and merges model data from both APIs into a local catalog JSON.

script: fetch_models.py

python scripts/fetch_models.py --output catalog.json
# OpenRouter only:
python scripts/fetch_models.py --or-only --output catalog.json
# Artificial Analysis only:
python scripts/fetch_models.py --aa-only --output catalog.json

Output fields per model:

analyze-model analyze-costs

Reads a usage log file and the catalog, then produces a cost breakdown by model with totals, percentages, and per-call averages. Resolves private-provider model IDs (e.g. opencode-go/*) to OpenRouter equivalents via references/aliases.json. Computes subscription efficiency for token-budget providers by comparing subscription cost vs. OpenRouter pay-per-token equivalent.

Agent pre-flight: Before running, check if subscription pricing is fresh:

python scripts/fetch_subscriptions.py

If the exit code is 1, dispatch subagents to research current pricing, then write references/subscriptions.json. Proceed once cache is fresh.

script: analyze_costs.py

python scripts/analyze_costs.py usage.json --catalog catalog.json
python scripts/analyze_costs.py usage.csv  --catalog catalog.json --output report.json

Usage log format — see references/usage-format.md

Output includes:

• Total USD spent
• Calls, input tokens, output tokens (global + per model)
• Cost % share per model (sorted by spend) — rendered as horizontal bars
• Unknown models that could not be priced
• Subscription efficiency — monthly-bucketed with per-provider cap logic
• Monthly sparklines — cap usage trends per provider
• Model jumping detection — sessions with model switches flagged
• Alias resolution — private models matched to OpenRouter catalog

analyze-model forecast

Projects future costs over a configurable horizon (default 30 days) using linear trend extrapolation from historical usage. Includes cheaper-model alternative recommendations.

script: forecast.py

python scripts/forecast.py usage.json --catalog catalog.json --days 30
python scripts/forecast.py usage.csv  --catalog catalog.json --days 90 --output forecast.json

Output includes:

• Projected total cost, calls, tokens for the horizon period (up to 365 days)
• Per-model projected cost with usage share — rendered as horizontal bars
• Multi-baseline projections — trend, worst-case, and average with comparison bars
• Up to 3 cheaper alternatives per top-cost model (requires aa enrichment in catalog)

analyze-model export-usage

Exports usage data from an LLM surface into the standard usage log format that analyze_costs.py and forecast.py consume.

script: export_usage.py

python scripts/export_usage.py --source opencode --output usage.json
python scripts/export_usage.py --source pi --output usage.json
python scripts/export_usage.py                    # interactive: pick a source
python scripts/export_usage.py --list-sources      # show available bridges
python scripts/export_usage.py --list-all          # include planned sources

How agents should use this: When a user asks about their LLM usage, costs, or forecasting, prompt them to select a source. Ask: "Which LLM surface are you using? I can extract from any supported surface (see --list-sources)." Then run the dispatcher with --source.

Typical Analysis Pipeline

# Step 0 (agent): refresh subscription pricing if stale
python scripts/fetch_subscriptions.py

# Step 1: Build the catalog
python scripts/fetch_models.py --output catalog.json

# Step 2: Export your usage from an LLM surface
python scripts/export_usage.py --source opencode --output usage.json
python scripts/export_usage.py --source pi --output usage.json
# (or manually provide a usage log — see references/usage-format.md)

# Step 3: Cost breakdown (now with aliases + subscription efficiency)
python scripts/analyze_costs.py usage.json --catalog catalog.json --output cost_report.json

# Step 4: 30-day forecast + alternatives
python scripts/forecast.py usage.json --catalog catalog.json --days 30 --output forecast.json

Quick Reference

Error Handling

References

Scripts

script: export_usage.py — Dispatcher for usage extraction from LLM surfaces. script: fetch_subscriptions.py — Cache-gate for subscription pricing (30-day TTL). script: chart_utils.py — ASCII chart rendering (horizontal bars, sparklines, gauges). script: bridges/opencode.py — SQLite usage bridge for that surface. script: bridges/pi.py — JSONL usage bridge for that surface (per-message usage + logged cost). script: bridges/base.py — Bridge protocol definition. script: client_openrouter.py — OpenRouter HTTP client (models endpoint). script: client_aa.py — Artificial Analysis HTTP client (free language/models/free endpoint). script: fetch_models.py — Fetches from both APIs and merges into a unified catalog. script: analyze_costs.py — Cost analysis engine; accepts JSON or CSV usage logs. script: forecast.py — Linear-trend forecasting + cheaper-alternative finder.