Data-Efficient Autoregressive-to-Diffusion Language Models via On-Policy Distillation

In three linesNew OPDLM method transforms autoregressive language models into diffusion models without full retraining. Via on-policy distillation, student model generates its own trajectories while frozen original model provides target logits. Result: 15x to 7,000x fewer training tokens required.

Summary generated by Claude — human-verified