Together AI Cost Tuning
Overview
Optimize Together AI costs with model selection, batching, and caching.
Instructions
Together AI Pricing Model
| Model Category |
Price (per 1M tokens) |
Example Models |
| Small (< 10B) |
$0.10-0.30 |
Llama-3.2-3B, Qwen-2.5-7B |
| Medium (10-40B) |
$0.60-1.20 |
Mixtral-8x7B, Llama-3.3-70B-Turbo |
| Large (40B+) |
$2.00-5.00 |
Llama-3.1-405B, DeepSeek-V3 |
| Image gen |
$0.003-0.05/image |
FLUX.1-schnell, SDXL |
| Embeddings |
$0.008/1M tokens |
M2-BERT |
| Fine-tuning |
~$5-25/hour |
Depends on model + GPU |
| Batch inference |
50% off |
Same models, async |
Cost Reduction Strategies
# 1. Use Turbo variants (faster, cheaper, similar quality)
# meta-llama/Llama-3.3-70B-Instruct-Turbo vs Llama-3.1-70B-Instruct
# 2. Batch inference (50% cost reduction)
batch_response = client.batch.create(
input_file_id=file_id,
model="meta-llama/Llama-3.3-70B-Instruct-Turbo",
completion_window="24h",
)
# 3. Cache responses for identical prompts
from functools import lru_cache
@lru_cache(maxsize=1000)
def cached_completion(prompt: str, model: str) -> str:
response = client.chat.completions.create(
model=model, messages=[{"role": "user", "content": prompt}],
)
return response.choices[0].message.content
# 4. Use smallest model that works
# Test with 3B first, upgrade to 70B only if quality insufficient
Error Handling
| Issue |
Cause |
Solution |
| High costs |
Wrong model tier |
Downsize model |
| Batch failures |
Invalid input format |
Validate JSONL |
| Fine-tuning expensive |
Too many epochs |
Start with 1-2 epochs |
Resources
Next Steps
For architecture patterns, see together-reference-architecture.
Source: jeremylongshore/claude-code-plugins-plus-skills → plugins/saas-packs/together-pack/skills/together-cost-tuning/SKILL.md
1---2name: together-cost-tuning3description: "'Together AI cost tuning for inference, fine-tuning, and model deployment. Use when working with Together AI''s OpenAI-compatible API. Trigger: \"together cost tuning\". '"4allowed-tools: "Read, Write, Edit, Bash(pip:*), Grep"5category: data-science-and-ml6source_repo: jeremylongshore/claude-code-plugins-plus-skills7source_path: "plugins/saas-packs/together-pack/skills/together-cost-tuning/SKILL.md"8source_url: https://github.com/jeremylongshore/claude-code-plugins-plus-skills/blob/HEAD/plugins/saas-packs/together-pack/skills/together-cost-tuning/SKILL.md9---1011# Together AI Cost Tuning1213## Overview1415Optimize Together AI costs with model selection, batching, and caching.1617## Instructions1819### Together AI Pricing Model2021| Model Category | Price (per 1M tokens) | Example Models |22|---------------|----------------------|----------------|23| Small (< 10B) | $0.10-0.30 | Llama-3.2-3B, Qwen-2.5-7B |24| Medium (10-40B) | $0.60-1.20 | Mixtral-8x7B, Llama-3.3-70B-Turbo |25| Large (40B+) | $2.00-5.00 | Llama-3.1-405B, DeepSeek-V3 |26| Image gen | $0.003-0.05/image | FLUX.1-schnell, SDXL |27| Embeddings | $0.008/1M tokens | M2-BERT |28| Fine-tuning | ~$5-25/hour | Depends on model + GPU |29| Batch inference | 50% off | Same models, async |3031### Cost Reduction Strategies3233```python34# 1. Use Turbo variants (faster, cheaper, similar quality)35# meta-llama/Llama-3.3-70B-Instruct-Turbo vs Llama-3.1-70B-Instruct3637# 2. Batch inference (50% cost reduction)38batch_response = client.batch.create(39 input_file_id=file_id,40 model="meta-llama/Llama-3.3-70B-Instruct-Turbo",41 completion_window="24h",42)4344# 3. Cache responses for identical prompts45from functools import lru_cache4647@lru_cache(maxsize=1000)48def cached_completion(prompt: str, model: str) -> str:49 response = client.chat.completions.create(50 model=model, messages=[{"role": "user", "content": prompt}],51 )52 return response.choices[0].message.content5354# 4. Use smallest model that works55# Test with 3B first, upgrade to 70B only if quality insufficient56```5758## Error Handling5960| Issue | Cause | Solution |61|-------|-------|----------|62| High costs | Wrong model tier | Downsize model |63| Batch failures | Invalid input format | Validate JSONL |64| Fine-tuning expensive | Too many epochs | Start with 1-2 epochs |6566## Resources6768- [Together AI Pricing](https://www.together.ai/pricing)69- [Batch Inference](https://docs.together.ai/docs/batch-inference)7071## Next Steps7273For architecture patterns, see `together-reference-architecture`.7475---7677**Source:** [`jeremylongshore/claude-code-plugins-plus-skills`](https://github.com/jeremylongshore/claude-code-plugins-plus-skills) → `plugins/saas-packs/together-pack/skills/together-cost-tuning/SKILL.md`
jeremylongshore