AI Token Improvement Plan Engineer
When to Use
- Audit where tokens are spent (by feature, model, tenant, turn type)
- Prioritize cost-saving initiatives with estimated savings and risk
- Draft a phased improvement plan with metrics and rollback criteria
- Define eval gates so cost cuts do not regress quality or safety
- Brief leadership on trade-offs (model tier, context size, agent depth)
When NOT to Use
- Implement context packing or compression code →
ai-context-engineer
- Rewrite one prompt or agent tool schema →
prompt-engineer
- Build RAG ingest, chunking, or agent runtime →
ai-engineer
- Weekly cost review ritual and release governance →
ai-lead-ops
- Memory store architecture →
ai-memory-developer
- Adversarial safety testing →
ai-redteam
- AI policy and regulatory mapping →
ai-risk-governance
Related skills
| Need |
Skill |
| Context budget and truncation |
ai-context-engineer |
| Prompt patterns and eval of wording |
prompt-engineer |
| RAG and agent implementation |
ai-engineer |
| Production ops and cost reviews |
ai-lead-ops |
| Memory write/read policy |
ai-memory-developer |
| Safety regression testing |
ai-redteam |
| AI solution architecture (commercial/enterprise) |
applied-ai-architect-commercial-enterprise |
| Token efficiency experiments and benchmarks |
research-engineer-scientist-tokens |
Core Workflows
1. Scope and baseline
Define:
- Surface area — chat, copilot, batch, agents, embeddings-only
- Time window — 7/30 days; exclude anomalies (launches, incidents)
- Segmentation — model, feature flag, tenant tier, environment
Collect baseline metrics (see references/token_audit.md).
Deliverable: current state table with top 5 cost drivers (≥70% of spend if possible).
2. Categorize spend
Bucket each driver:
| Category |
Examples |
| Input bloat |
Long system prompts, duplicated docs, full chat history |
| Output bloat |
Verbose defaults, no max_tokens, unconstrained agents |
| Model choice |
Opus-class for simple classification |
| Retrieval |
Over-fetching chunks, huge tool results in context |
| Agent loops |
Extra tool rounds, retry storms |
| Infrastructure |
Re-embedding unchanged corpora, log payloads in prompts |
Tag fixed per request vs scales with users/sessions.
3. Initiative backlog
For each idea record:
- Description and owner team
- Estimated savings (% tokens or $/month) with assumptions
- Effort (S/M/L) and dependencies
- Quality risk (low/med/high) and required evals
- Measurement — metric that proves success
Prioritize with impact × confidence ÷ effort; never ship without eval plan.
See references/improvement_backlog.md and references/optimization_playbook.md.
4. Quality and safety guardrails
Every initiative must list:
- Golden-set evals (task accuracy, format)
- Safety set (if user-facing)
- Latency check (p95)
- Rollback trigger (e.g., CSAT drop, eval regression >X%)
See references/measurement_and_kpis.md.
5. Phased plan
Typical phases:
| Phase |
Focus |
Duration |
| 0 — Measure |
Instrumentation, dashboards, attribution |
1–2 weeks |
| 1 — Quick wins |
max_tokens, model routing, prompt trim, cache |
2–4 weeks |
| 2 — Structure |
RAG top-k, history compression, tool output limits |
4–8 weeks |
| 3 — Architecture |
Agent budget caps, routing policies, batch/offline |
8+ weeks |
Each phase: goals, initiatives, owners, exit criteria.
See references/rollout_plan.md.
6. Executive summary
## Situation — spend and growth rate
## Target — $ or tokens/session goal by date
## Top drivers — ranked list
## Plan — 3–5 initiatives per phase
## Risks — quality, latency, eng capacity
## Asks — budget, headcount, eval time
Hand ongoing ops to ai-lead-ops after plan approval.
When to load references
- Baseline and attribution →
references/token_audit.md
- Backlog template and scoring →
references/improvement_backlog.md
- Technique catalog →
references/optimization_playbook.md
- KPIs and eval gates →
references/measurement_and_kpis.md
- Phasing and rollback →
references/rollout_plan.md
1---2name: ai-token-improvement-plan-engineer3description: Guides creation of AI token and cost improvement plans—baseline audits, spend attribution, optimization initiative backlog (prompt, context, model routing, RAG, agents), impact estimates, quality guardrails, measurement KPIs, and phased rollout with owners. Use when building a token reduction roadmap, cost optimization program, LLM unit-economics improvement plan, or executive brief on cutting inference spend without breaking evals—not for hands-on context layout implementation (ai-context-engineer), single-prompt rewrites (prompt-engineer), RAG pipeline build (ai-engineer), or AI ops cadence and vendor governance (ai-lead-ops). For end-to-end commercial/enterprise AI solution architecture before cost programs, use applied-ai-architect-commercial-enterprise. Empirical token studies and ablations: research-engineer-scientist-tokens.4---56# AI Token Improvement Plan Engineer78## When to Use910- Audit where tokens are spent (by feature, model, tenant, turn type)11- Prioritize cost-saving initiatives with estimated savings and risk12- Draft a phased improvement plan with metrics and rollback criteria13- Define eval gates so cost cuts do not regress quality or safety14- Brief leadership on trade-offs (model tier, context size, agent depth)1516## When NOT to Use1718- Implement context packing or compression code → `ai-context-engineer`19- Rewrite one prompt or agent tool schema → `prompt-engineer`20- Build RAG ingest, chunking, or agent runtime → `ai-engineer`21- Weekly cost review ritual and release governance → `ai-lead-ops`22- Memory store architecture → `ai-memory-developer`23- Adversarial safety testing → `ai-redteam`24- AI policy and regulatory mapping → `ai-risk-governance`2526## Related skills2728| Need | Skill |29|---|---|30| Context budget and truncation | `ai-context-engineer` |31| Prompt patterns and eval of wording | `prompt-engineer` |32| RAG and agent implementation | `ai-engineer` |33| Production ops and cost reviews | `ai-lead-ops` |34| Memory write/read policy | `ai-memory-developer` |35| Safety regression testing | `ai-redteam` |36| AI solution architecture (commercial/enterprise) | `applied-ai-architect-commercial-enterprise` |37| Token efficiency experiments and benchmarks | `research-engineer-scientist-tokens` |3839## Core Workflows4041### 1. Scope and baseline4243Define:4445- **Surface area** — chat, copilot, batch, agents, embeddings-only46- **Time window** — 7/30 days; exclude anomalies (launches, incidents)47- **Segmentation** — model, feature flag, tenant tier, environment4849Collect baseline metrics (see `references/token_audit.md`).5051Deliverable: **current state** table with top 5 cost drivers (≥70% of spend if possible).5253### 2. Categorize spend5455Bucket each driver:5657| Category | Examples |58|---|---|59| Input bloat | Long system prompts, duplicated docs, full chat history |60| Output bloat | Verbose defaults, no max_tokens, unconstrained agents |61| Model choice | Opus-class for simple classification |62| Retrieval | Over-fetching chunks, huge tool results in context |63| Agent loops | Extra tool rounds, retry storms |64| Infrastructure | Re-embedding unchanged corpora, log payloads in prompts |6566Tag **fixed per request** vs **scales with users/sessions**.6768### 3. Initiative backlog6970For each idea record:7172- Description and owner team73- **Estimated savings** (% tokens or $/month) with assumptions74- **Effort** (S/M/L) and dependencies75- **Quality risk** (low/med/high) and required evals76- **Measurement** — metric that proves success7778Prioritize with impact × confidence ÷ effort; never ship without eval plan.7980**See `references/improvement_backlog.md` and `references/optimization_playbook.md`.**8182### 4. Quality and safety guardrails8384Every initiative must list:8586- Golden-set evals (task accuracy, format)87- Safety set (if user-facing)88- Latency check (p95)89- Rollback trigger (e.g., CSAT drop, eval regression >X%)9091**See `references/measurement_and_kpis.md`.**9293### 5. Phased plan9495Typical phases:9697| Phase | Focus | Duration |98|---|---|---|99| 0 — Measure | Instrumentation, dashboards, attribution | 1–2 weeks |100| 1 — Quick wins | max_tokens, model routing, prompt trim, cache | 2–4 weeks |101| 2 — Structure | RAG top-k, history compression, tool output limits | 4–8 weeks |102| 3 — Architecture | Agent budget caps, routing policies, batch/offline | 8+ weeks |103104Each phase: goals, initiatives, owners, exit criteria.105106**See `references/rollout_plan.md`.**107108### 6. Executive summary109110```markdown111## Situation — spend and growth rate112## Target — $ or tokens/session goal by date113## Top drivers — ranked list114## Plan — 3–5 initiatives per phase115## Risks — quality, latency, eng capacity116## Asks — budget, headcount, eval time117```118119Hand ongoing ops to `ai-lead-ops` after plan approval.120121## When to load references122123- **Baseline and attribution** → `references/token_audit.md`124- **Backlog template and scoring** → `references/improvement_backlog.md`125- **Technique catalog** → `references/optimization_playbook.md`126- **KPIs and eval gates** → `references/measurement_and_kpis.md`127- **Phasing and rollback** → `references/rollout_plan.md`
daemon-blockint-tech