Research Engineer / Research Scientist, Tokens
When to Use
- Frame research questions on tokens, context length, or inference cost
- Design experiments with baselines, ablations, and statistical rigor
- Build benchmarks for tokens-per-successful-task, effective context, cache leverage
- Measure tokenizer and formatting effects on length and model behavior
- Evaluate compression, summarization, routing, or distillation for token savings
- Analyze long-context phenomena (needle, lost-in-middle, attention budget)
- Write research memos with reproducible methods and honest limitations
- Translate findings into actionable thresholds for engineering and product
When NOT to Use
- Executive token reduction program with phased rollout →
ai-token-improvement-plan-engineer
- Implement context packing, compaction code paths →
ai-context-engineer
- Rewrite one production prompt →
prompt-engineer
- General literature survey unrelated to tokens →
ai-researcher
- Production RAG/agent deployment →
ai-engineer
- Classical ML without LLM token focus →
data-scientist
Related skills
| Need |
Skill |
| General research methodology |
ai-researcher |
| Cost improvement program / roadmap |
ai-token-improvement-plan-engineer |
| Production context assembly |
ai-context-engineer |
| Prompt wording and eval harness |
prompt-engineer |
| RAG and agent runtime build |
ai-engineer |
| Statistical testing and cohort analysis |
data-scientist |
| Adversarial robustness of compressed context |
ai-redteam |
| Commercial AI architecture |
applied-ai-architect-commercial-enterprise |
Core Workflows
1. Research framing (tokens)
Hypothesis, metrics, baselines, budget.
See references/research_framing_tokens.md.
2. Measurement and instrumentation
Token accounting, logging, fair comparison.
See references/measurement_instrumentation.md.
3. Experiment design and ablations
Controls, sweeps, power, stopping rules.
See references/experiment_design_ablations.md.
4. Context, tokenization, and long-context
Tokenizer, placement, window effects.
See references/context_tokenization_longcontext.md.
5. Compression and efficiency methods
Summarization, routing, distillation research.
See references/compression_efficiency_methods.md.
6. Reproducibility and research reporting
Memos, artifacts, handoff to engineering.
See references/reproducibility_reporting.md.
Outputs
- Pre-registration / experiment plan — hypothesis, metrics, stop criteria
- Results table — mean ± CI; tokens and quality side by side
- Pareto chart narrative — quality vs tokens at operating points
- Ablation appendix — what mattered, what did not
- Research memo — conclusion, limits, recommended next build
- Artifact bundle — configs, seeds, eval scripts, hashed datasets
Principles
- Report tokens and quality together — never optimize one without the other
- Match tokenizer and model — counts from the deployment tokenizer/API
- Control confounds — temperature, system prompt, tool schemas held fixed across arms
- Pre-register primary metric — avoid p-hacking across slice metrics
- Separate science from rollout — research recommends;
ai-token-improvement-plan-engineer owns program
1---2name: research-engineer-scientist-tokens3description: Guides research engineering and science on LLM tokens—hypotheses about context use, tokenization, compression, and inference efficiency; rigorous benchmarks (tokens per task, quality–cost Pareto); ablation design; instrumentation and reproducible logs; and research memos that inform product decisions. Use when designing token-efficiency experiments, measuring context utilization, comparing compression or routing methods, analyzing tokenizer effects, or writing technical reports on token/cost trade-offs—not for phased cost roadmaps and owners (ai-token-improvement-plan-engineer), production context pipeline implementation (ai-context-engineer), single-prompt edits (prompt-engineer), general non-token AI research (ai-researcher), or shipping features (ai-engineer).4---56# Research Engineer / Research Scientist, Tokens78## When to Use910- Frame **research questions** on tokens, context length, or inference cost11- Design **experiments** with baselines, ablations, and statistical rigor12- Build **benchmarks** for tokens-per-successful-task, effective context, cache leverage13- Measure **tokenizer and formatting** effects on length and model behavior14- Evaluate **compression, summarization, routing, or distillation** for token savings15- Analyze **long-context** phenomena (needle, lost-in-middle, attention budget)16- Write **research memos** with reproducible methods and honest limitations17- Translate findings into **actionable thresholds** for engineering and product1819## When NOT to Use2021- Executive token reduction program with phased rollout → `ai-token-improvement-plan-engineer`22- Implement context packing, compaction code paths → `ai-context-engineer`23- Rewrite one production prompt → `prompt-engineer`24- General literature survey unrelated to tokens → `ai-researcher`25- Production RAG/agent deployment → `ai-engineer`26- Classical ML without LLM token focus → `data-scientist`2728## Related skills2930| Need | Skill |31|---|---|32| General research methodology | `ai-researcher` |33| Cost improvement program / roadmap | `ai-token-improvement-plan-engineer` |34| Production context assembly | `ai-context-engineer` |35| Prompt wording and eval harness | `prompt-engineer` |36| RAG and agent runtime build | `ai-engineer` |37| Statistical testing and cohort analysis | `data-scientist` |38| Adversarial robustness of compressed context | `ai-redteam` |39| Commercial AI architecture | `applied-ai-architect-commercial-enterprise` |4041## Core Workflows4243### 1. Research framing (tokens)4445Hypothesis, metrics, baselines, budget.4647**See `references/research_framing_tokens.md`.**4849### 2. Measurement and instrumentation5051Token accounting, logging, fair comparison.5253**See `references/measurement_instrumentation.md`.**5455### 3. Experiment design and ablations5657Controls, sweeps, power, stopping rules.5859**See `references/experiment_design_ablations.md`.**6061### 4. Context, tokenization, and long-context6263Tokenizer, placement, window effects.6465**See `references/context_tokenization_longcontext.md`.**6667### 5. Compression and efficiency methods6869Summarization, routing, distillation research.7071**See `references/compression_efficiency_methods.md`.**7273### 6. Reproducibility and research reporting7475Memos, artifacts, handoff to engineering.7677**See `references/reproducibility_reporting.md`.**7879## Outputs8081- **Pre-registration / experiment plan** — hypothesis, metrics, stop criteria82- **Results table** — mean ± CI; tokens and quality side by side83- **Pareto chart narrative** — quality vs tokens at operating points84- **Ablation appendix** — what mattered, what did not85- **Research memo** — conclusion, limits, recommended next build86- **Artifact bundle** — configs, seeds, eval scripts, hashed datasets8788## Principles8990- **Report tokens and quality together** — never optimize one without the other91- **Match tokenizer and model** — counts from the deployment tokenizer/API92- **Control confounds** — temperature, system prompt, tool schemas held fixed across arms93- **Pre-register primary metric** — avoid p-hacking across slice metrics94- **Separate science from rollout** — research recommends; `ai-token-improvement-plan-engineer` owns program
daemon-blockint-tech