# Diffusionvl Ar To Diffusion

> Convert pre-trained autoregressive vision-language models into diffusion VLMs without architectural modifications. Use block diffusion strategy enabling arbitrary-length generation and KV-cache reuse. Hybrid attention enforces bidirectional within blocks, causal between blocks. Requires less than 5% of data compared to prior diffusion VLM methods.

- Skill: `adu2021/diffusionvl-ar-to-diffusion` (Agent Skill)
- Install (CLI): `npx skillmds@latest add adu2021/diffusionvl-ar-to-diffusion`
- Raw SKILL.md: https://api.skillmd.com/api/skills/adu2021/diffusionvl-ar-to-diffusion/raw
- Safety review: pending
- Works with: Claude Code, Claude.ai, OpenAI Codex
- Category: AI & ML
- License: MIT
- Author: adu2021 (https://skillmd.com/u/adu2021)
- Updated: 2026-09-17
- Page: https://skillmd.com/skills/adu2021/diffusionvl-ar-to-diffusion

---


## Skill Summary

DiffusionVL demonstrates direct conversion of already vision-language aligned autoregressive models into diffusion VLMs through full-parameter diffusion finetuning. The approach converts the "next-token prediction paradigm into a diffusion paradigm." Block diffusion strategy enables arbitrary-length generation with intra-block parallel denoising and inter-block autoregressive decoding. Hybrid attention mechanism enforces bidirectional attention within blocks and causal attention between blocks. Remarkably, requires less than 5% of data compared to prior diffusion VLM methods, proving "the gap between dVLMs and AR-VLMs is minimal."

## When To Use

- Converting existing AR vision-language models to parallel-generation diffusion models
- Scenarios requiring efficient VLM inference with minimal retraining
- Projects where parallel generation benefits outweigh minor quality trade-offs
- Research on paradigm-agnostic model conversion techniques

## When NOT To Use

- Building VLMs from scratch (end-to-end diffusion training may be preferable)
- Applications where AR properties are critical (causal attention constraints)
- Scenarios with very limited training data (even 5% is substantial for some projects)
- Domains where hybrid attention mechanism causes artifacts

## Core Technique

Two main pathways enable vision-language paradigm conversion:

**1. AR-VLM to dVLM (Paradigm Shift)**
Direct conversion of already vision-language aligned autoregressive models through full-parameter diffusion finetuning. Convert the "next-token prediction paradigm into a diffusion paradigm" using minimal data.

**2. AR-LM to dVLM (Modality + Paradigm Shift)**
Two-stage approach: connector first aligns vision and text spaces using autoregressive training, then diffusion finetuning completes the conversion. Enables VLM creation from vision and language separately.

**3. Block Diffusion Strategy**
Enable arbitrary-length generation and KV-cache reuse through:
- Intra-block parallel denoising: generate block contents in parallel
- Inter-block autoregressive decoding: generate blocks sequentially

**4. Hybrid Attention Mechanism**
Enforce bidirectional attention within blocks (full context for parallel denoising) and causal attention between blocks (respecting generation order). This balances efficiency with coherence.

## Implementation Notes

Start with pre-trained AR-VLM. Implement block diffusion strategy with hybrid attention pattern. Fine-tune entire model with diffusion objective using your target data (only 5% of original AR training needed). Monitor accuracy preservation and measure parallel generation speedup. Validate on your target VLM tasks.

## References

- Original paper: DiffusionVL (Dec 2025)
- Block diffusion language models
- Vision-language model architectures