# Research Engineer Scientist Tokens

> Guides research engineering and science on LLM tokens—hypotheses about context use, tokenization, compression, and inference efficiency; rigorous benchmarks (tokens per task, quality–cost Pareto); ablation design; instrumentation and reproducible logs; and research memos that inform product decisions. Use when designing token-efficiency experiments, measuring context utilization, comparing compression or routing methods, analyzing tokenizer effects, or writing technical reports on token/cost trade-offs—not for phased cost roadmaps and owners (ai-token-improvement-plan-engineer), production context pipeline implementation (ai-context-engineer), single-prompt edits (prompt-engineer), general non-token AI research (ai-researcher), or shipping features (ai-engineer).

- Skill: `daemon-blockint-tech/research-engineer-scientist-tokens` (Agent Skill, multi-file: 7 files)
- Install (CLI): `npx skillmds@latest add daemon-blockint-tech/research-engineer-scientist-tokens`
- Raw SKILL.md: https://api.skillmd.com/api/skills/daemon-blockint-tech/research-engineer-scientist-tokens/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: DevOps & Infra
- Author: daemon-blockint-tech (https://skillmd.com/u/daemon-blockint-tech)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/daemon-blockint-tech/research-engineer-scientist-tokens

---


# Research Engineer / Research Scientist, Tokens

## When to Use

- Frame **research questions** on tokens, context length, or inference cost
- Design **experiments** with baselines, ablations, and statistical rigor
- Build **benchmarks** for tokens-per-successful-task, effective context, cache leverage
- Measure **tokenizer and formatting** effects on length and model behavior
- Evaluate **compression, summarization, routing, or distillation** for token savings
- Analyze **long-context** phenomena (needle, lost-in-middle, attention budget)
- Write **research memos** with reproducible methods and honest limitations
- Translate findings into **actionable thresholds** for engineering and product

## When NOT to Use

- Executive token reduction program with phased rollout → `ai-token-improvement-plan-engineer`
- Implement context packing, compaction code paths → `ai-context-engineer`
- Rewrite one production prompt → `prompt-engineer`
- General literature survey unrelated to tokens → `ai-researcher`
- Production RAG/agent deployment → `ai-engineer`
- Classical ML without LLM token focus → `data-scientist`

## Related skills

| Need | Skill |
|---|---|
| General research methodology | `ai-researcher` |
| Cost improvement program / roadmap | `ai-token-improvement-plan-engineer` |
| Production context assembly | `ai-context-engineer` |
| Prompt wording and eval harness | `prompt-engineer` |
| RAG and agent runtime build | `ai-engineer` |
| Statistical testing and cohort analysis | `data-scientist` |
| Adversarial robustness of compressed context | `ai-redteam` |
| Commercial AI architecture | `applied-ai-architect-commercial-enterprise` |

## Core Workflows

### 1. Research framing (tokens)

Hypothesis, metrics, baselines, budget.

**See `references/research_framing_tokens.md`.**

### 2. Measurement and instrumentation

Token accounting, logging, fair comparison.

**See `references/measurement_instrumentation.md`.**

### 3. Experiment design and ablations

Controls, sweeps, power, stopping rules.

**See `references/experiment_design_ablations.md`.**

### 4. Context, tokenization, and long-context

Tokenizer, placement, window effects.

**See `references/context_tokenization_longcontext.md`.**

### 5. Compression and efficiency methods

Summarization, routing, distillation research.

**See `references/compression_efficiency_methods.md`.**

### 6. Reproducibility and research reporting

Memos, artifacts, handoff to engineering.

**See `references/reproducibility_reporting.md`.**

## Outputs

- **Pre-registration / experiment plan** — hypothesis, metrics, stop criteria
- **Results table** — mean ± CI; tokens and quality side by side
- **Pareto chart narrative** — quality vs tokens at operating points
- **Ablation appendix** — what mattered, what did not
- **Research memo** — conclusion, limits, recommended next build
- **Artifact bundle** — configs, seeds, eval scripts, hashed datasets

## Principles

- **Report tokens and quality together** — never optimize one without the other
- **Match tokenizer and model** — counts from the deployment tokenizer/API
- **Control confounds** — temperature, system prompt, tool schemas held fixed across arms
- **Pre-register primary metric** — avoid p-hacking across slice metrics
- **Separate science from rollout** — research recommends; `ai-token-improvement-plan-engineer` owns program