Drifting Objectives for Refining Discrete Diffusion Language Models

In three linesTokenDrift applies drifting methods (objective correction) to discrete diffusion language models. The technique lifts categorical predictions to soft-token features, applies anti-symmetric drifting in a frozen semantic space, and backpropagates to DDLM logits. On MDLM and DUO, TokenDrift reduces generation perplexity by 89% and 86% at 4 NFE.

Summary generated by Claude — human-verified