# Deer Diffusion Speculative Decoding

> Enable efficient speculative decoding by training discrete diffusion language models for parallel draft generation. Use AR-style distillation and scribe refinement to train dLLMs. Eliminate left-to-right error accumulation through independent parallel proposals. Achieve 5.54× speedup on HumanEval vs. 2.41× for AR-based methods.

- Skill: `adu2021/deer-diffusion-speculative-decoding` (Agent Skill)
- Install (CLI): `npx skillmds@latest add adu2021/deer-diffusion-speculative-decoding`
- Raw SKILL.md: https://api.skillmd.com/api/skills/adu2021/deer-diffusion-speculative-decoding/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- License: MIT
- Author: adu2021 (https://skillmd.com/u/adu2021)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/adu2021/deer-diffusion-speculative-decoding

---


## Skill Summary

DEER (Draft with diffusion, vErify with autoRegressive) introduces speculative decoding using discrete diffusion language models for efficient parallel draft generation. The approach trains dLLMs through two-stage alignment: AR-style distillation enabling prefix-conditioned continuation and scribe refinement sharpening predictions near verification boundaries. Unlike AR drafters suffering left-to-right uncertainty accumulation, DEER's parallel generation makes "the proposal at position i independent of previously drafted tokens," enabling acceptance lengths up to 32 tokens vs. ~10 for competing methods. Results show 5.54× speedup on HumanEval.

## When To Use

- Building efficient LLM inference systems requiring high acceptance rates
- Scenarios where speculative decoding speedups are critical for latency
- Projects where parallel draft generation prevents error accumulation
- Research on efficient alternatives to autoregressive drafting

## When NOT To Use

- Applications already meeting latency goals with simpler methods
- Scenarios where training discrete diffusion models is computationally infeasible
- Domains where AR drafters' lower acceptance rates aren't performance bottlenecks
- Models with strict parameter budgets for additional draft model

## Core Technique

Two-stage alignment pipeline trains diffusion drafters:

**1. AR-Style Distillation (Stage I)**
Train discrete diffusion language model (dLLM) for prefix-conditioned continuation. Learn from truncated teacher answers marked with SEP token, enabling model to generate coherent suffixes given fixed prefix. This bridges AR teacher and diffusion student paradigms.

**2. Scribe Refinement (Stage II)**
Enhance accuracy near verification boundary through "weighted suffix masking with exponentially decaying loss," focusing training on tokens most critical for acceptance. Improve predictions most likely to affect speculative decoding outcome.

**3. Parallel Generation Advantage**
Unlike AR drafters with left-to-right uncertainty accumulation, dLLM generates entire token blocks in single denoising step. "The proposal at position i is independent of previously drafted tokens," preventing error propagation and enabling acceptance lengths up to 32 tokens versus ~10 for competing methods.

## Speedup Results

- 5.54× speedup on HumanEval vs. EAGLE-3's 2.41×
- Substantially better than AR-based speculative decoding approaches
- Maintains competitive verification overhead

## Implementation Notes

Start with pretrained teacher AR model. Train discrete diffusion LM for parallel generation conditioned on prefixes. Implement AR-style distillation with SEP token marking. Add scribe refinement focusing on boundary tokens. Integrate into verifier framework for speculative decoding. Measure acceptance rates and overall speedup.

## References

- Original paper: DEER (Dec 2025)
- Discrete diffusion language models
- Speculative decoding frameworks