Models

AI Model Selection Rules

Core Principle

• No single model is best for everything — match model to task, not brand loyalty
• A $0.75/M model often performs identically to a $40/M model for simple tasks
• Test cheaper alternatives before committing to expensive defaults

Cost Reality

• Output tokens cost 3-10x more than input tokens — advertised input prices are misleading
• Calculate real cost with your actual input/output ratio, not theoretical pricing
• Batch/async APIs offer 50% discounts — use them for non-real-time workloads
• Prompt caching reduces repeated context costs significantly

Task Matching

Coding

• Architecture and design decisions: Use frontier models (Opus-class) — they catch subtle issues cheaper models miss
• Day-to-day implementation: Mid-tier models (Sonnet-class) offer 90% of capability at 20% of cost
• Parallel subtasks and scaffolding: Fast/cheap models (Haiku-class) — speed matters more than depth
• Code review: Thorough models catch async bugs and edge cases that fast models miss

Non-Coding

• Complex reasoning and math: Extended thinking modes justify their cost for hard problems
• General assistance: User preference studies favor models different from benchmark leaders
• High-volume simple queries: Cheapest models perform identically — don't overpay
• Long documents: Context window size determines viability — some offer 1M+ tokens

Claude Code vs Codex CLI

• Claude Code: Fast iteration, UI/frontend, interactive debugging — developer stays in the loop
• Codex CLI: Long-running background tasks, large refactors, set-and-forget — accuracy over speed
• Both tools have value — use Claude Code for implementation, Codex for final review
• File size limits differ — Claude Code struggles with files over 25K tokens

Orchestration Pattern

• Planning phase: Use expensive/smart models to break down problems correctly
• Execution phase: Use balanced models, parallelize where possible
• Review phase: Use accurate models for final verification — catches bugs others miss
• This pattern beats using one model for everything at similar total cost

Benchmark Skepticism

• Benchmark scores vary 2-3x based on scaffolding and evaluation method
• User preference rankings differ significantly from benchmark rankings
• SWE-bench scores don't predict real-world coding quality reliably
• Models drift week-to-week — last month's best may underperform today

Open Source Viability

• DeepSeek and similar models approach frontier performance at 1/50th API cost
• Self-hosting eliminates API rate limits and price variability
• MIT/Apache licensed models allow commercial use without restrictions
• Consider for: data privacy, cost predictability, custom fine-tuning

Model Selection Mistakes

• Using premium models for chatbot responses that cheap models handle identically
• Ignoring context window limits — chunking long documents costs more than using large-context models
• Expecting consistency — same prompt gives different results over time as models update
• Trusting speed over accuracy for complex tasks — fast models trade thoroughness for latency

Practical Guidelines

• Default to mid-tier for most tasks, escalate to frontier only when quality suffers
• Track actual costs per workflow, not just per-token rates
• Build verification into pipelines — don't trust any model blindly
• Reassess model choices quarterly — pricing and capabilities shift constantly