# Unmasking Diffusion Policies

> Learn which tokens to unmask during diffusion sampling via reinforcement learning instead of heuristics. Policies eliminate manual tuning and scale across block sizes—crucial when semi-autoregressive generation needs dynamic, learned unmasking strategies.

- Skill: `adu2021/unmasking-diffusion-policies` (Agent Skill)
- Install (CLI): `npx skillmds@latest add adu2021/unmasking-diffusion-policies`
- Raw SKILL.md: https://api.skillmd.com/api/skills/adu2021/unmasking-diffusion-policies/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- License: MIT
- Author: adu2021 (https://skillmd.com/u/adu2021)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/adu2021/unmasking-diffusion-policies

---


## Overview

Rather than relying on manual confidence thresholds for token unmasking, this approach casts diffusion language model sampling as an MDP where a lightweight transformer policy learns optimal unmasking decisions based on token confidences.

## When to Use

- Diffusion language model inference optimization
- Semi-autoregressive generation with block-based sampling
- Need for dynamic unmasking beyond confidence thresholds
- Scaling across different block sizes
- Learning unmasking strategies instead of manual tuning

## When NOT to Use

- Scenarios where heuristic thresholds work adequately
- Autoregressive decoding without masking
- Tasks not benefiting from RL optimization

## Core Technique

RL-based policy for unmasking token selection:

```python
# Learned unmasking policies for dLLMs
class UnmaskingPolicy:
    def __init__(self):
        self.policy = nn.Sequential(
            nn.Linear(vocab_size, 256),
            nn.ReLU(),
            nn.Linear(256, vocab_size)  # Unmasking logits
        )

    def learn_unmasking_policy(self, dllm, dataset):
        """Train policy via RL on dLLM sampling."""
        for batch in dataset:
            # Generate with dLLM
            confidences = dllm.get_token_confidences(batch)

            # Policy decides which tokens to unmask
            unmasking_logits = self.policy(confidences)
            unmasking_probs = torch.softmax(unmasking_logits, dim=-1)

            # Sample unmasking decisions
            decisions = torch.multinomial(unmasking_probs, num_samples=1)

            # Apply unmasking
            sampled_tokens = dllm.sample_with_unmasking(decisions)

            # Reward: similarity to full diffusion output
            full_output = dllm.full_diffusion_sampling(batch)
            reward = compute_similarity(sampled_tokens, full_output)

            # Policy gradient
            loss = -reward * torch.log(unmasking_probs[decisions])
            loss.backward()

        self.optimizer.step()

    def unmask_during_inference(self, confidences):
        """Apply learned policy for token selection."""
        unmasking_logits = self.policy(confidences)
        decisions = torch.argmax(unmasking_logits, dim=-1)
        return decisions
```

## Key Results

- Learned policies match/exceed heuristic thresholds
- Scalability across block sizes
- Eliminates hyperparameter tuning

## References

- Original paper: https://arxiv.org/abs/2512.09106
- Focus: Optimal unmasking in diffusion LLMs
- Domain: Diffusion models, sampling strategies