# Mtp General

> Implement, update, or review a general multi-token prediction (MTP) scheme for transformer LMs, including multi-head token-shift inputs, per-head targets, fusion blocks (norm + concat + projector + transformer layer), and inference-time multi-token decoding.

- Skill: `happenmass/mtp-general` (Agent Skill, multi-file: 2 files)
- Install (CLI): `npx skillmds@latest add happenmass/mtp-general`
- Raw SKILL.md: https://api.skillmd.com/api/skills/happenmass/mtp-general/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- Author: Happenmass (https://skillmd.com/u/happenmass)
- Updated: 2026-09-22
- Page: https://skillmd.com/skills/happenmass/mtp-general

---


# General MTP (Multi-Token Prediction)

Use this skill when designing or debugging a general MTP implementation (not tied to a specific project).

## Workflow

- Read `references/mtp-general.md` for the current design, inputs/targets, and inference flow.
- Keep training and inference consistent: head0 uses the base LLM output; subsequent heads use MTP fusion blocks with shifted speech tokens and eos alignment.
- When changing the MTP structure, update both `prepare_lm_input_target` and inference logic together.
- If integrating into a project, ensure config/registry points to the new MTP class and model selection logic supports it.

## Guardrails

- Preserve token alignment rules: each extra head removes one speech token and appends one eos token to keep lengths aligned.
- Do not add extra eos to head0 inputs (targets already include eos).
- Keep module grouping: MTP fusion block encapsulates norm, projector, and decoder layer.