# Eagle3 New Model

> Add a new model to the EAGLE3 offline pipeline. Generates an hf_offline_eagle3.yaml launcher config for a new model checkpoint, choosing the right hidden state dump backend (TRT-LLM / HF / vLLM) and GPU configuration. Use when user wants to run EAGLE3 on a model that does not yet have a YAML in tools/launcher/examples/ or asks how to configure the pipeline for a new checkpoint.

- Skill: `nvidia/eagle3-new-model` (Agent Skill)
- Install (CLI): `npx skillmds@latest add nvidia/eagle3-new-model`
- Raw SKILL.md: https://api.skillmd.com/api/skills/nvidia/eagle3-new-model/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: DevOps & Infra
- Author: NVIDIA (https://skillmd.com/u/nvidia)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/nvidia/eagle3-new-model

---


# EAGLE3 New Model Configuration

Create `tools/launcher/examples/<Org>/<Model>/hf_offline_eagle3.yaml` by **copying the
closest existing example and adapting it**. Pick a reference with the same shape as the
target (dense vs MoE, similar size) from `tools/launcher/examples/` — e.g. the Qwen3-8B
config for a dense model.

The pipeline is a 4-task config (`task_0` data synthesis → `task_1` hidden-state dump →
`task_2` train → `task_3` benchmark). The task structure, args, containers, and GPU/node
sizing are all visible in the existing examples — infer them from a reference rather than
hand-rolling. This file documents only the two things that are **not** obvious from the
examples: which dump backend to pick, and the model-specific gotchas.

## Choosing the `task_1` hidden-state dump backend

| Backend | Script | When to use |
|---------|--------|-------------|
| vLLM | `common/eagle3/dump_offline_data_vllm.sh` | **Default.** Broad coverage via vLLM's native hidden-state extractor. |
| HF | `common/eagle3/dump_offline_data_hf.sh` | VLMs / multimodal, custom-code models, sliding-window attention (TRT-LLM can't serve these). |
| TRT-LLM | `common/eagle3/dump_offline_data.sh` | Pure-text models with TRT-LLM support; pass `--tp <TP>` and `--moe-ep <EP>`. |

Rule of thumb: **HF** if the model is a VLM or uses sliding-window attention; **vLLM**
otherwise. TRT-LLM only when you specifically want its kernels for a supported plain-text model.

## Model-specific adjustments

These are the non-obvious knobs that vary per model:

| Situation | What to change |
|---|---|
| Requires `--trust-remote-code` | Add to `task_0` vLLM args (before the `--` separator) and to `task_3` benchmark args |
| MoE with large expert hidden dim | Increase `intermediate_size` in `eagle_config.json` to match `moe_intermediate_size` |
| Custom tokenizer (e.g. tiktoken) | Set `TIKTOKEN_RS_CACHE_DIR` env var in `task_0` and `task_1` |

After adapting the config, preview it with `--dryrun` before submitting.